{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T19:53:13Z","timestamp":1772913193888,"version":"3.50.1"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2022,12,22]],"date-time":"2022-12-22T00:00:00Z","timestamp":1671667200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,12,22]],"date-time":"2022-12-22T00:00:00Z","timestamp":1671667200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Introduction<\/jats:title>\n                <jats:p>Detecting safety signals attributed to a drug in scientific literature is a fundamental issue in pharmacovigilance. The constant increase in the volume of publications requires the automation of this tedious task, in order to find and extract relevant articles from the pack. This task is critical, as serious Adverse Drug Reactions (ADRs) still account for a large number of hospital admissions each year.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Objectives<\/jats:title>\n                <jats:p>The aim of this study is to develop an augmented intelligence methodology for automatically identifying relevant publications mentioning an established link between a Drug and a Serious Adverse Event, according to the European Medicines Agency (EMA) definition of seriousness.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>The proposed pipeline, called LiSA (for Literature Search Application), is based on three independent deep learning models supporting a precise detection of safety signals in the biomedical literature. By combining a Bidirectional Encoder Representations from Transformers (BERT) algorithms and a modular architecture, the pipeline achieves a precision of 0.81 and a recall of 0.89 at sentences level in articles extracted from PubMed (either abstract or full-text). We also measured that by using LiSA, a medical reviewer increases by a factor of 2.5 the number of relevant documents it can collect and evaluate compared to a simple keyword search. In the interest of re-usability, emphasis was placed on building a modular pipeline allowing the insertion of other NLP modules to enrich the results provided by the system, and extend it to other use cases. In addition, a lightweight visualization tool was developed to analyze and monitor safety signal results.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>Overall, the generic pipeline and the visualization tool proposed in this article allows for efficient and accurate monitoring of serious adverse drug reactions from the literature and can easily be adapted to similar pharmacovigilance use cases. To facilitate reproducibility and benefit other research studies, we also shared a first benchmark dataset for Serious Adverse Drug Events detection.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s12911-022-02085-0","type":"journal-article","created":{"date-parts":[[2022,12,22]],"date-time":"2022-12-22T12:04:49Z","timestamp":1671710689000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["LiSA: an assisted literature search pipeline for detecting serious adverse drug events with deep learning"],"prefix":"10.1186","volume":"22","author":[{"given":"Vincent","family":"Martenot","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Valentin","family":"Masdeu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jean","family":"Cupe","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Faustine","family":"Gehin","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Margot","family":"Blanchon","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Julien","family":"Dauriat","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alexander","family":"Horst","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael","family":"Renaudin","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Philippe","family":"Girard","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jean-Daniel","family":"Zucker","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2022,12,22]]},"reference":[{"key":"2085_CR1","doi-asserted-by":"publisher","unstructured":"van\u00a0de Schoot R, de Bruin J, Schram R, Zahedi P, de Boer J, Weijdema F, Kramer B, Huijts M, Hoogerwerf M, Ferdinands G, Harkema A, Willemsen J, Ma Y, Fang Q, Hindriks S, Tummers L, Oberski DL. An open source machine learning framework for efficient and transparent systematic reviews. Nature Machine Intelligence 2021;3(2):125\u2013133. https:\/\/doi.org\/10.1038\/s42256-020-00287-7. Bandiera_abtest: a Cc_license_type: cc_by Cg_type: Nature Research Journals Number: 2 Primary_atype: Research Publisher: Nature Publishing Group Subject_term: Computational biology and bioinformatics;Computer science;Medical research;SARS-CoV-2 Subject_term_id: computational-biology-and-bioinformatics;computer-science;medical-research;sars-cov-2","DOI":"10.1038\/s42256-020-00287-7"},{"key":"2085_CR2","doi-asserted-by":"publisher","unstructured":"Piccinni C, Poluzzi E, Orsini M, Bergamaschi S. PV-OWL \u2013 Pharmacovigilance surveillance through semantic web-based platform for continuous and integrated monitoring of drug-related adverse effects in open data sources and social media. 2017 IEEE 3rd International Forum on Research and Technologies for Society and Industry (RTSI) (2017). https:\/\/doi.org\/10.1109\/RTSI.2017.8065931.","DOI":"10.1109\/RTSI.2017.8065931"},{"key":"2085_CR3","unstructured":"Brisebois R, Abran A, Nadembega A, N\u2019techobo P. An Assisted Literature Review using Machine Learning Models to Recommend a Relevant Reference Papers List, 2017;24."},{"key":"2085_CR4","doi-asserted-by":"publisher","unstructured":"Alimova I, Tutubalina E. Detecting Adverse Drug Reactions from Biomedical Texts with Neural Networks. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop, 2019;415\u2013421. Association for Computational Linguistics, Florence, Italy. https:\/\/doi.org\/10.18653\/v1\/P19-2058. https:\/\/www.aclweb.org\/anthology\/P19-2058","DOI":"10.18653\/v1\/P19-2058"},{"issue":"1","key":"2085_CR5","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2019.102131","volume":"57","author":"B Fan","year":"2020","unstructured":"Fan B, Fan W, Smith C, Garner H. Adverse drug event detection and extraction from open data: a deep learning approach. Inf Process Manage. 2020;57(1): 102131. https:\/\/doi.org\/10.1016\/j.ipm.2019.102131.","journal-title":"Inf Process Manage"},{"key":"2085_CR6","doi-asserted-by":"publisher","unstructured":"Eberts M, Ulges A. Span-based Joint Entity and Relation Extraction with Transformer Pre-training. arXiv:1909.07755 [cs] 2021. https:\/\/doi.org\/10.3233\/FAIA200321","DOI":"10.3233\/FAIA200321"},{"key":"2085_CR7","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/ACCESS.2018.2882443","volume":"PP","author":"P Ding","year":"2018","unstructured":"Ding P, Zhou X, Zhang X, Wang J, Lei Z. An attentive neural sequence labeling model for adverse drug reactions mentions extraction. IEEE Access. 2018;PP:1\u20131. https:\/\/doi.org\/10.1109\/ACCESS.2018.2882443.","journal-title":"IEEE Access"},{"key":"2085_CR8","doi-asserted-by":"publisher","unstructured":"Zhang S, Dev S, Voyles J, Rao AS. Attention-Based Multi-Task Learning in Pharmacovigilance. In: 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), 2018, pp. 2324\u201322328. https:\/\/doi.org\/10.1109\/BIBM.2018.8621286.","DOI":"10.1109\/BIBM.2018.8621286"},{"key":"2085_CR9","doi-asserted-by":"publisher","unstructured":"Yuwen L, Chen S, Zhang H. Detecting Potential Serious Adverse Drug Reactions Using Sequential Pattern Mining Method. In: 2018 IEEE 9th International Conference on Software Engineering and Service Science (ICSESS), 2018, pp. 56\u201359. https:\/\/doi.org\/10.1109\/ICSESS.2018.8663856","DOI":"10.1109\/ICSESS.2018.8663856"},{"issue":"1","key":"2085_CR10","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1007\/s40264-019-00869-4","volume":"43","author":"R Routray","year":"2020","unstructured":"Routray R, Tetarenko N, Abu-Assal C, Mockute R, Assuncao B, Chen H, Bao S, Danysz K, Desai S, Cicirello S, Willis V, Alford SH, Krishnamurthy V, Mingle E. Application of augmented intelligence for pharmacovigilance case seriousness determination. Drug Saf. 2020;43(1):57\u201366. https:\/\/doi.org\/10.1007\/s40264-019-00869-4.","journal-title":"Drug Saf"},{"key":"2085_CR11","unstructured":"FDA: FAERS (FDA Adverse Event Reporting System). [Online; Accessed 11 Dec 2021]. https:\/\/open.fda.gov\/data\/faers\/."},{"key":"2085_CR12","unstructured":"FDA: MedWatch: The FDA Safety Information and Adverse Event Reporting Program. [Online; Accessed 11 Dec 2021]. https:\/\/www.fda.gov\/safety\/medwatch-fda-safety-information-and-adverse-event-reporting-program."},{"key":"2085_CR13","doi-asserted-by":"crossref","unstructured":"Dunne C, Shneiderman B, Gove R, Klavans J, Dorr B. Rapid understanding of scientific paper collections: integrating statistics, text analysis, and visualization (2011).","DOI":"10.1002\/asi.22652"},{"key":"2085_CR14","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1186\/1472-6947-14-13","volume":"14","author":"S Yeleswarapu","year":"2014","unstructured":"Yeleswarapu S, Rao A, Joseph T, Saipradeep VG, Srinivasan R. A pipeline to extract drug-adverse event pairs from multiple data sources. BMC Med Inform Decis Mak. 2014;14:13. https:\/\/doi.org\/10.1186\/1472-6947-14-13.","journal-title":"BMC Med Inform Decis Mak"},{"key":"2085_CR15","doi-asserted-by":"publisher","unstructured":"Gattepaille LM. Using the WHO database of spontaneous reports to build joint vector representations of drugs and adverse drug reactions, a promising avenue for pharmacovigilance. In: 2019 IEEE International Conference on Healthcare Informatics (ICHI), 2019:1\u20136. https:\/\/doi.org\/10.1109\/ICHI.2019.8904551.","DOI":"10.1109\/ICHI.2019.8904551"},{"issue":"1","key":"2085_CR16","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1186\/1472-6947-14-91","volume":"14","author":"H Sampathkumar","year":"2014","unstructured":"Sampathkumar H, Chen X-W, Luo B. Mining Adverse Drug Reactions from online healthcare forums using Hidden Markov Model. BMC Med Inform Decis Mak. 2014;14(1):91. https:\/\/doi.org\/10.1186\/1472-6947-14-91.","journal-title":"BMC Med Inform Decis Mak"},{"issue":"3","key":"2085_CR17","doi-asserted-by":"publisher","first-page":"413","DOI":"10.1136\/amiajnl-2012-000930","volume":"20","author":"R Harpaz","year":"2013","unstructured":"Harpaz R, Vilar S, Dumouchel W, Salmasian H, Haerian K, Shah NH, Chase HS, Friedman C. Combing signals from spontaneous reports and electronic health records for detection of adverse drug reactions. JAMIA. 2013;20(3):413\u20139. https:\/\/doi.org\/10.1136\/amiajnl-2012-000930.","journal-title":"JAMIA"},{"issue":"5","key":"2085_CR18","doi-asserted-by":"publisher","first-page":"885","DOI":"10.1016\/j.jbi.2012.04.008","volume":"45","author":"H Gurulingappa","year":"2012","unstructured":"Gurulingappa H, Rajput AM, Roberts A, Fluck J, Hofmann-Apitius M, Toldo L. Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports. J Biomed Inform. 2012;45(5):885\u201392. https:\/\/doi.org\/10.1016\/j.jbi.2012.04.008.","journal-title":"J Biomed Inform"},{"key":"2085_CR19","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1186\/2041-1480-3-15","volume":"3","author":"H Gurulingappa","year":"2012","unstructured":"Gurulingappa H, Mateen-Rajput A, Toldo L. Extraction of potential adverse drug events from medical case reports. J Biomed Semant. 2012;3:15. https:\/\/doi.org\/10.1186\/2041-1480-3-15.","journal-title":"J Biomed Semant"},{"key":"2085_CR20","unstructured":"ChEMBL: ChEMBL Is a Manually curated database of bioactive molecules with drug-like properties. [Online; Accessed 11 Dec 2021]. https:\/\/www.ebi.ac.uk\/chembl\/."},{"key":"2085_CR21","unstructured":"MedDRA: Medical Dictionary for Regulatory Activities. [Online; Accessed 11 Dec 2021]. https:\/\/www.meddra.org\/."},{"key":"2085_CR22","doi-asserted-by":"crossref","unstructured":"Pappagari R, \u017belasko P, Villalba J, Carmiel Y, Dehak N. Hierarchical Transformers for Long Document Classification. arXiv:1910.10781 [cs, stat] 2019. arXiv: 1910.10781","DOI":"10.1109\/ASRU46091.2019.9003958"},{"issue":"6","key":"2085_CR23","doi-asserted-by":"publisher","first-page":"333","DOI":"10.1145\/360825.360855","volume":"18","author":"AV Aho","year":"1975","unstructured":"Aho AV, Corasick MJ. Efficient string matching: an aid to bibliographic search. Commun ACM. 1975;18(6):333\u201340. https:\/\/doi.org\/10.1145\/360825.360855.","journal-title":"Commun ACM"},{"key":"2085_CR24","unstructured":"EMA: ICH E2A Clinical Safety Data Management: Definitions and Standards for Expedited Reporting. [Online; Accessed 11 Dec 2021]. https:\/\/www.fda.gov\/safety\/reporting-serious-problems-fda\/what-serious-adverse-event."},{"key":"2085_CR25","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2021.103983","author":"Z Yuan","year":"2022","unstructured":"Yuan Z, Zhao Z, Sun H, Li J, Wang F, Yu S. Coder: Knowledge-infused cross-lingual medical term embedding for term normalization. J Biomed Inf. 2022. https:\/\/doi.org\/10.1016\/j.jbi.2021.103983.","journal-title":"J Biomed Inf"},{"issue":"4","key":"2085_CR26","doi-asserted-by":"publisher","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","volume":"36","author":"J Lee","year":"2019","unstructured":"Lee J, Yoon W, Kim S, Kim D, Kim S, So CH, Kang J. BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics. 2019;36(4):1234\u201340. https:\/\/doi.org\/10.1093\/bioinformatics\/btz682. arxiv:1901.08746.","journal-title":"Bioinformatics"},{"key":"2085_CR27","doi-asserted-by":"crossref","unstructured":"Peng Y, Yan S, Lu Z. Transfer learning in biomedical natural language processing: An evaluation of bert and elmo on ten benchmarking datasets. In: Proceedings of the 2019 Workshop on Biomedical Natural Language Processing (BioNLP 2019), (2019), pp. 58\u201365.","DOI":"10.18653\/v1\/W19-5006"},{"key":"2085_CR28","doi-asserted-by":"crossref","unstructured":"Beltagy I, Lo K, Cohan A. Scibert: A pretrained language model for scientific text. In: EMNLP. Association for Computational Linguistics? (2019). https:\/\/www.aclweb.org\/anthology\/D19-1371.","DOI":"10.18653\/v1\/D19-1371"},{"key":"2085_CR29","doi-asserted-by":"crossref","unstructured":"Alsentzer E, Murphy JR, Boag W, Weng W, Jin D, Naumann T, McDermott MBA. Publicly available clinical BERT embeddings. CoRR abs\/1904.03323 2019. arxiv:1904.03323.","DOI":"10.18653\/v1\/W19-1909"},{"key":"2085_CR30","unstructured":"Devlin J, Chang M, Lee K, Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR abs\/1810.04805 (2018). arxiv:1810.04805."},{"key":"2085_CR31","unstructured":"Gu Y, Tinn R, Cheng H, Lucas M, Usuyama N, Liu X, Naumann T, Gao J, Poon H. Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing 2020. arxiv:2007.15779."},{"issue":"1","key":"2085_CR32","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1093\/jamia\/ocv044","volume":"23","author":"IJ Marshall","year":"2016","unstructured":"Marshall IJ, Kuiper J, Wallace BC. RobotReviewer: evaluation of a system for automatically assessing bias in clinical trials. J Am Med Inform Assoc. 2016;23(1):193\u2013201.","journal-title":"J Am Med Inform Assoc"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-022-02085-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-022-02085-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-022-02085-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,22]],"date-time":"2022-12-22T12:08:04Z","timestamp":1671710884000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-022-02085-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,22]]},"references-count":32,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["2085"],"URL":"https:\/\/doi.org\/10.1186\/s12911-022-02085-0","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,22]]},"assertion":[{"value":"28 June 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 December 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 December 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing Interests"}}],"article-number":"338"}}