{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T18:01:42Z","timestamp":1772906502446,"version":"3.50.1"},"reference-count":46,"publisher":"Oxford University Press (OUP)","issue":"8","license":[{"start":{"date-parts":[[2023,8,23]],"date-time":"2023-08-23T00:00:00Z","timestamp":1692748800000},"content-version":"vor","delay-in-days":22,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,8,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Motivation<\/jats:title>\n                  <jats:p>The process of drug discovery is notoriously complex, costing an average of 2.6 billion dollars and taking \u223c13\u2009years to bring a new drug to the market. The success rate for new drugs is alarmingly low (around 0.0001%), and severe adverse drug reactions (ADRs) frequently occur, some of which may even result in death. Early identification of potential ADRs is critical to improve the efficiency and safety of the drug development process.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>In this study, we employed pretrained large language models (LLMs) to predict the likelihood of a drug being withdrawn from the market due to safety concerns. Our method achieved an area under the curve (AUC) of over 0.75 through cross-database validation, outperforming classical machine learning models and graph-based models. Notably, our pretrained LLMs successfully identified over 50% drugs that were subsequently withdrawn, when predictions were made on a subset of drugs with inconsistent labeling between the training and test sets.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Availability and implementation<\/jats:title>\n                  <jats:p>The code and datasets are available at https:\/\/github.com\/eyalmazuz\/DrugWithdrawn.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btad519","type":"journal-article","created":{"date-parts":[[2023,8,22]],"date-time":"2023-08-22T15:11:16Z","timestamp":1692717076000},"source":"Crossref","is-referenced-by-count":11,"title":["Pretrained transformer models for predicting the withdrawal of drugs from the market"],"prefix":"10.1093","volume":"39","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-1077-1736","authenticated-orcid":false,"given":"Eyal","family":"Mazuz","sequence":"first","affiliation":[{"name":"Department of Software and Information Systems Engineering, Ben-Gurion University of the Negev , P.O.B. 653 , Beer-Sheva, 8410501, Israel"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0647-8753","authenticated-orcid":false,"given":"Guy","family":"Shtar","sequence":"additional","affiliation":[{"name":"Department of Software and Information Systems Engineering, Ben-Gurion University of the Negev , P.O.B. 653 , Beer-Sheva, 8410501, Israel"}]},{"given":"Nir","family":"Kutsky","sequence":"additional","affiliation":[{"name":"Department of Software and Information Systems Engineering, Ben-Gurion University of the Negev , P.O.B. 653 , Beer-Sheva, 8410501, Israel"}]},{"given":"Lior","family":"Rokach","sequence":"additional","affiliation":[{"name":"Department of Software and Information Systems Engineering, Ben-Gurion University of the Negev , P.O.B. 653 , Beer-Sheva, 8410501, Israel"}]},{"given":"Bracha","family":"Shapira","sequence":"additional","affiliation":[{"name":"Department of Software and Information Systems Engineering, Ben-Gurion University of the Negev , P.O.B. 653 , Beer-Sheva, 8410501, Israel"}]}],"member":"286","published-online":{"date-parts":[[2023,8,23]]},"reference":[{"key":"2023083107053711800_btad519-B1","author":"Ahmad","year":"2022"},{"key":"2023083107053711800_btad519-B2","doi-asserted-by":"crossref","first-page":"2064","DOI":"10.1021\/acs.jcim.1c00600","article-title":"Molgpt: molecular generation using a transformer-decoder model","volume":"62","author":"Bagal","year":"2021","journal-title":"J Chem Inf Model"},{"key":"2023083107053711800_btad519-B3","first-page":"785","author":"Chen","year":"2016"},{"key":"2023083107053711800_btad519-B4","doi-asserted-by":"crossref","first-page":"4977","DOI":"10.1021\/jm4004285","article-title":"QSAR modeling: where have you been? where are you going to?","volume":"57","author":"Cherkasov","year":"2014","journal-title":"J Med Chem"},{"key":"2023083107053711800_btad519-B5","author":"Chithrananda","year":"2020"},{"key":"2023083107053711800_btad519-B6","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1007\/BF00994018","article-title":"Support-vector networks","volume":"20","author":"Cortes","year":"1995","journal-title":"Mach Learn"},{"key":"2023083107053711800_btad519-B7","doi-asserted-by":"crossref","first-page":"694","DOI":"10.1177\/0092861512458776","article-title":"Are pharmaceutical market withdrawals preventable? a preliminary analysis","volume":"46","author":"Daggumalli","year":"2012","journal-title":"Drug Information J"},{"key":"2023083107053711800_btad519-B8","author":"Devlin","year":"2018"},{"key":"2023083107053711800_btad519-B9","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1016\/j.jhealeco.2016.01.012","article-title":"Innovation in the pharmaceutical industry: new estimates of R&D costs","volume":"47","author":"DiMasi","year":"2016","journal-title":"J Health Econ"},{"key":"2023083107053711800_btad519-B10","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1021\/acs.jcim.7b00616","article-title":"Mol2vec: unsupervised machine learning approach with chemical intuition","volume":"58","author":"Jaeger","year":"2018","journal-title":"J Chem Inf Model"},{"key":"2023083107053711800_btad519-B11","doi-asserted-by":"crossref","first-page":"165","DOI":"10.1016\/j.cbi.2006.12.006","article-title":"Computer-aided drug discovery and development (CADDD): in silico-chemico-biological approach","volume":"171","author":"Kapetanovic","year":"2008","journal-title":"Chem Biol Interact"},{"key":"2023083107053711800_btad519-B12","doi-asserted-by":"crossref","first-page":"D1102","DOI":"10.1093\/nar\/gky1033","article-title":"Pubchem 2019 update: improved access to chemical data","volume":"47","author":"Kim","year":"2019","journal-title":"Nucleic Acids Res"},{"key":"2023083107053711800_btad519-B13","doi-asserted-by":"crossref","first-page":"e106298","DOI":"10.1371\/journal.pone.0106298","article-title":"Adverse drug reaction prediction using scores produced by large-scale drug-protein target docking on high-performance computing machines","volume":"9","author":"LaBute","year":"2014","journal-title":"PloS One"},{"key":"2023083107053711800_btad519-B14","author":"Landrum","year":"2006"},{"key":"2023083107053711800_btad519-B15","doi-asserted-by":"crossref","first-page":"2215","DOI":"10.1001\/jama.287.17.2215","article-title":"Timing of new black box warnings and withdrawals for prescription medications","volume":"287","author":"Lasser","year":"2002","journal-title":"JAMA"},{"key":"2023083107053711800_btad519-B16","doi-asserted-by":"crossref","first-page":"1200","DOI":"10.1001\/jama.279.15.1200","article-title":"Incidence of adverse drug reactions in hospitalized patients: a meta-analysis of prospective studies","volume":"279","author":"Lazarou","year":"1998","journal-title":"JAMA"},{"key":"2023083107053711800_btad519-B17","first-page":"1","volume-title":"Noncovalent Forces","author":"Leszczynski","year":"2006"},{"key":"2023083107053711800_btad519-B18","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1136\/amiajnl-2013-002051","article-title":"Determining molecular predictors of adverse drug reactions with causality analysis based on structure learning","volume":"21","author":"Liu","year":"2014","journal-title":"J Am Med Inform Assoc"},{"key":"2023083107053711800_btad519-B19","author":"Liu","year":"2019"},{"key":"2023083107053711800_btad519-B20","first-page":"10012","author":"Liu","year":"2021"},{"key":"2023083107053711800_btad519-B21","doi-asserted-by":"crossref","first-page":"1376","DOI":"10.1021\/acs.jcim.1c01467","article-title":"Unified deep learning model for multitask reaction predictions with explanation","volume":"62","author":"Lu","year":"2022","journal-title":"J Chem Inf Model"},{"key":"2023083107053711800_btad519-B22","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1124\/pr.110.003533","article-title":"Pharmacogenetics, pharmacogenomics, and individualized medicine","volume":"63","author":"Ma","year":"2011","journal-title":"Pharmacol Rev"},{"key":"2023083107053711800_btad519-B23","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1517\/14740338.2012.631910","article-title":"Epidemiology and characteristics of adverse drug reactions caused by drug\u2013drug interactions","volume":"11","author":"Magro","year":"2012","journal-title":"Expert Opin Drug Saf"},{"key":"2023083107053711800_btad519-B24","doi-asserted-by":"crossref","first-page":"D930","DOI":"10.1093\/nar\/gky1075","article-title":"Chembl: towards direct deposition of bioassay data","volume":"47","author":"Mendez","year":"2019","journal-title":"Nucleic Acids Res"},{"key":"2023083107053711800_btad519-B25","author":"NCATS","year":"2020"},{"key":"2023083107053711800_btad519-B26","author":"Ninan","year":"2012"},{"key":"2023083107053711800_btad519-B27","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1186\/s12916-016-0553-2","article-title":"Post-marketing withdrawal of 462 medicinal products because of adverse drug reactions: a systematic review of the world literature","volume":"14","author":"Onakpoya","year":"2016","journal-title":"BMC Med"},{"key":"2023083107053711800_btad519-B28","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1016\/j.cmpb.2017.02.004","article-title":"Classification of nervous system withdrawn and approved drugs with toxprint features via machine learning strategies","volume":"142","author":"Onay","year":"2017","journal-title":"Comput Methods Programs Biomed"},{"key":"2023083107053711800_btad519-B29","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1186\/1471-2105-12-169","article-title":"Predicting drug side-effect profiles: a chemical fragment-based approach","volume":"12","author":"Pauwels","year":"2011","journal-title":"BMC Bioinformatics"},{"key":"2023083107053711800_btad519-B30","author":"Puzyn","year":"2010"},{"key":"2023083107053711800_btad519-B31","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel","year":"2020","journal-title":"J. Mach. Learn. Res"},{"key":"2023083107053711800_btad519-B32","volume-title":"Deep Learning for the Life Sciences","author":"Ramsundar","year":"2019"},{"key":"2023083107053711800_btad519-B33","doi-asserted-by":"crossref","first-page":"308","DOI":"10.1021\/ci800344p","article-title":"Gaining insight into off-target mediated effects of drug candidates with a comprehensive systems chemical biology analysis","volume":"49","author":"Scheiber","year":"2009","journal-title":"J Chem Inf Model"},{"key":"2023083107053711800_btad519-B34","author":"Shah","year":"2006"},{"key":"2023083107053711800_btad519-B35","doi-asserted-by":"crossref","first-page":"862","DOI":"10.1038\/nature03197","article-title":"Virtual screening of chemical libraries","volume":"432","author":"Shoichet","year":"2004","journal-title":"Nature"},{"key":"2023083107053711800_btad519-B36","first-page":"1115","author":"Shtar","year":"2021"},{"key":"2023083107053711800_btad519-B37","doi-asserted-by":"crossref","first-page":"D1080","DOI":"10.1093\/nar\/gkv1192","article-title":"Withdrawn\u2014a resource for withdrawn and discontinued drugs","volume":"44","author":"Siramshetty","year":"2016","journal-title":"Nucleic Acids Res"},{"key":"2023083107053711800_btad519-B38","doi-asserted-by":"crossref","first-page":"125ra31","DOI":"10.1126\/scitranslmed.3003377","article-title":"Data-driven prediction of drug effects and interactions","volume":"4","author":"Tatonetti","year":"2012","journal-title":"Sci Transl Med"},{"key":"2023083107053711800_btad519-B39","article-title":"Attention is all you need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Adv Neural Inf Process Syst"},{"key":"2023083107053711800_btad519-B40","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1021\/ci00057a005","article-title":"Smiles, a chemical language and information system. 1. introduction to methodology and encoding rules","volume":"28","author":"Weininger","year":"1988","journal-title":"J Chem Inf Comput Sci"},{"key":"2023083107053711800_btad519-B41","doi-asserted-by":"crossref","first-page":"1421","DOI":"10.1016\/S1359-6446(05)03632-9","article-title":"Keynote review: in vitro safety pharmacology profiling: an essential tool for successful drug development","volume":"10","author":"Whitebread","year":"2005","journal-title":"Drug Discov Today"},{"key":"2023083107053711800_btad519-B42","doi-asserted-by":"crossref","first-page":"2211","DOI":"10.1056\/NEJMra032424","article-title":"Drug metabolism and variability among patients in drug response","volume":"352","author":"Wilkinson","year":"2005","journal-title":"N Engl J Med"},{"key":"2023083107053711800_btad519-B43","doi-asserted-by":"crossref","first-page":"D1074","DOI":"10.1093\/nar\/gkx1037","article-title":"Drugbank 5.0: a major update to the drugbank database for 2018","volume":"46","author":"Wishart","year":"2018","journal-title":"Nucleic Acids Res"},{"key":"2023083107053711800_btad519-B44","doi-asserted-by":"crossref","first-page":"1363","DOI":"10.1001\/archinte.165.12.1363","article-title":"Adverse drug event surveillance and drug withdrawals in the United States, 1969-2002: the importance of reporting suspected reactions","volume":"165","author":"Wysowski","year":"2005","journal-title":"Arch Intern Med"},{"key":"2023083107053711800_btad519-B45","doi-asserted-by":"crossref","first-page":"510","DOI":"10.1021\/ci500667v","article-title":"New publicly available chemical query language, CSRML, to support chemotype representations for application to data mining and modeling","volume":"55","author":"Yang","year":"2015","journal-title":"J Chem Inf Model"},{"key":"2023083107053711800_btad519-B46","doi-asserted-by":"crossref","first-page":"3370","DOI":"10.1021\/acs.jcim.9b00237","article-title":"Analyzing learned molecular representations for property prediction","volume":"59","author":"Yang","year":"2019","journal-title":"J Chem Inf Model"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btad519\/51234008\/btad519.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/39\/8\/btad519\/51312553\/btad519.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/39\/8\/btad519\/51312553\/btad519.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,31]],"date-time":"2023-08-31T07:06:08Z","timestamp":1693465568000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/doi\/10.1093\/bioinformatics\/btad519\/7248908"}},"subtitle":[],"editor":[{"given":"Jonathan","family":"Wren","sequence":"additional","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2023,8,1]]},"references-count":46,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2023,8,1]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btad519","relation":{},"ISSN":["1367-4811"],"issn-type":[{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,8,1]]},"published":{"date-parts":[[2023,8,1]]},"article-number":"btad519"}}