{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,16]],"date-time":"2026-05-16T14:13:11Z","timestamp":1778940791316,"version":"3.51.4"},"reference-count":49,"publisher":"Oxford University Press (OUP)","issue":"Supplement_2","license":[{"start":{"date-parts":[[2022,9,1]],"date-time":"2022-09-01T00:00:00Z","timestamp":1661990400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/501100004410","name":"Scientific and Technological Research Council of Turkey","doi-asserted-by":"publisher","award":["ECCB2022"],"award-info":[{"award-number":["ECCB2022"]}],"id":[{"id":"10.13039\/501100004410","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004410","name":"Scientific and Technological Research Council of Turkey","doi-asserted-by":"publisher","award":["119E133"],"award-info":[{"award-number":["119E133"]}],"id":[{"id":"10.13039\/501100004410","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004410","name":"Scientific and Technological Research Council of Turkey","doi-asserted-by":"publisher","award":["2211-A"],"award-info":[{"award-number":["2211-A"]}],"id":[{"id":"10.13039\/501100004410","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,9,16]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Motivation<\/jats:title><jats:p>The development of novel compounds targeting proteins of interest is one of the most important tasks in the pharmaceutical industry. Deep generative models have been applied to targeted molecular design and have shown promising results. Recently, target-specific molecule generation has been viewed as a translation between the protein language and the chemical language. However, such a model is limited by the availability of interacting protein\u2013ligand pairs. On the other hand, large amounts of unlabelled protein sequences and chemical compounds are available and have been used to train language models that learn useful representations. In this study, we propose exploiting pretrained biochemical language models to initialize (i.e. warm start) targeted molecule generation models. We investigate two warm start strategies: (i) a one-stage strategy where the initialized model is trained on targeted molecule generation and (ii) a two-stage strategy containing a pre-finetuning on molecular generation followed by target-specific training. We also compare two decoding strategies to generate compounds: beam search and sampling.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>The results show that the warm-started models perform better than a baseline model trained from scratch. The two proposed warm-start strategies achieve similar results to each other with respect to widely used metrics from benchmarks. However, docking evaluation of the generated compounds for a number of novel proteins suggests that the one-stage strategy generalizes better than the two-stage strategy. Additionally, we observe that beam search outperforms sampling in both docking evaluation and benchmark metrics for assessing compound quality.<\/jats:p><\/jats:sec><jats:sec><jats:title>Availability and implementation<\/jats:title><jats:p>The source code is available at https:\/\/github.com\/boun-tabi\/biochemical-lms-for-drug-design and the materials (i.e., data, models, and outputs) are archived in Zenodo at https:\/\/doi.org\/10.5281\/zenodo.6832145.<\/jats:p><\/jats:sec><jats:sec><jats:title>Supplementary information<\/jats:title><jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p><\/jats:sec>","DOI":"10.1093\/bioinformatics\/btac482","type":"journal-article","created":{"date-parts":[[2022,9,20]],"date-time":"2022-09-20T09:15:04Z","timestamp":1663665304000},"page":"ii155-ii161","source":"Crossref","is-referenced-by-count":16,"title":["Exploiting pretrained biochemical language models for targeted drug design"],"prefix":"10.1093","volume":"38","author":[{"given":"G\u00f6k\u00e7e","family":"Uludo\u011fan","sequence":"first","affiliation":[{"name":"Department of Computer Engineering, Bo\u011fazi\u00e7i University , \u0130stanbul 34342, Turkey"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Elif","family":"Ozkirimli","sequence":"additional","affiliation":[{"name":"Data and Analytics Chapter, Pharma International Informatics, F. Hoffmann-La Roche AG 4303, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kutlu O","family":"Ulgen","sequence":"additional","affiliation":[{"name":"Department of Chemical Engineering, Bo\u011fazi\u00e7i University , \u0130stanbul 34342, Turkey"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nilg\u00fcn","family":"Karal\u0131","sequence":"additional","affiliation":[{"name":"Faculty of Pharmacy, Department of Pharmaceutical Chemistry, \u0130stanbul University , \u0130stanbul 34116, Turkey"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Arzucan","family":"\u00d6zg\u00fcr","sequence":"additional","affiliation":[{"name":"Department of Computer Engineering, Bo\u011fazi\u00e7i University , \u0130stanbul 34342, Turkey"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2022,9,18]]},"reference":[{"key":"2023041408000651700_","first-page":"13642","author":"Arroyo","year":"2021"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"3577","DOI":"10.1038\/s41598-019-38746-w","article-title":"Probabilistic variable-length segmentation of protein sequences for discriminative motif discovery (DiMotif) and sequence embedding (ProtVecX)","volume":"9","author":"Asgari","year":"2019","journal-title":"Sci. Rep"},{"key":"2023041408000651700_","article-title":"Latent molecular optimization for targeted therapeutic design","author":"Aumentado-Armstrong","year":"2018","journal-title":"arXiv"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"2887","DOI":"10.1021\/jm9602928","article-title":"The properties of known drugs. 1. Molecular frameworks","volume":"39","author":"Bemis","year":"1996","journal-title":"J. Med. Chem"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"25024","DOI":"10.1088\/2632-2153\/abe808","article-title":"Data-driven molecular design for discovery and synthesis of novel ligands: a case study on SARS-CoV-2","volume":"2","author":"Born","year":"2021","journal-title":"Mach. Learn. Sci. Technol"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"D520","DOI":"10.1093\/nar\/gky949","article-title":"Protein data bank: the single global archive for 3D macromolecular structure data","volume":"47","author":"Burley","year":"2019","journal-title":"Nucleic Acids Res"},{"key":"2023041408000651700_","article-title":"Target-specific and selective drug design for covid-19 using deep generative models","author":"Chenthamarakshan","year":"2020","journal-title":"arXiv"},{"key":"2023041408000651700_","article-title":"Chemberta: large-scale self-supervised pretraining for molecular property prediction","author":"Chithrananda","year":"2020","journal-title":"arXiv"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"1422","DOI":"10.1093\/bioinformatics\/btp163","article-title":"Biopython: freely available python tools for computational molecular biology and bioinformatics","volume":"25","author":"Cock","year":"2009","journal-title":"Bioinformatics"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"1503","DOI":"10.1002\/cmdc.200800178","article-title":"On the art of compiling and using \u2018drug-like\u2019 chemical fragment spaces","volume":"3","author":"Degen","year":"2008","journal-title":"ChemMedChem"},{"key":"2023041408000651700_","first-page":"82","article-title":"Pymol: an open-source molecular graphics tool","volume":"40","author":"DeLano","year":"2002","journal-title":"CCP4 Newsl. Protein Crystallogr"},{"key":"2023041408000651700_","first-page":"4171","author":"Devlin","year":"2019"},{"key":"2023041408000651700_","article-title":"Is map decoding all you need? The inadequacy of the mode in neural machine translation","author":"Eikema","year":"2020","journal-title":"arXiv"},{"key":"2023041408000651700_","article-title":"Prottrans: towards cracking the language of life\u2019s code through self-supervised deep learning and high performance computing","author":"Elnaggar","year":"2020","journal-title":"arXiv"},{"key":"2023041408000651700_","article-title":"Molecular representation learning with language models and domain-relevant auxiliary tasks","author":"Fabian","year":"2020","journal-title":"arXiv"},{"key":"2023041408000651700_","article-title":"Pre-training protein language models with label-agnostic binding pairs enhances performance in downstream tasks","author":"Filipavicius","year":"2020","journal-title":"arXiv"},{"key":"2023041408000651700_","first-page":"3371","article-title":"Interpretable drug target prediction using deep neural representation","author":"Gao","year":"2018"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"D1045","DOI":"10.1093\/nar\/gkv1072","article-title":"BindingDB in 2015: a public database for medicinal chemistry, computational chemistry and systems pharmacology","volume":"44","author":"Gilson","year":"2016","journal-title":"Nucleic Acids Res"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-020-79682-4","article-title":"Transformer neural network for protein-specific de novo drug generation as a machine translation problem","volume":"11","author":"Grechishnikova","year":"2021","journal-title":"Sci. Rep"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"298","DOI":"10.1002\/cpt.1846","article-title":"Does in vitro potency predict clinically efficacious concentrations?","volume":"108","author":"Jansson-L\u00f6fmark","year":"2020","journal-title":"Clin. Pharmacol. Ther"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"D1102","DOI":"10.1093\/nar\/gky1033","article-title":"Pubchem 2019 update: improved access to chemical data","volume":"47","author":"Kim","year":"2019","journal-title":"Nucleic Acids Res"},{"key":"2023041408000651700_","first-page":"47","article-title":"Ancestral Gumbel-top-k sampling for sampling without replacement","volume":"21","author":"Kool","year":"2020","journal-title":"J. Mach. Learn. Res"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"636","DOI":"10.1038\/nrd2378","article-title":"The application of discovery toxicology and pathology towards the design of safer pharmaceutical lead candidates","volume":"6","author":"Kramer","year":"2007","journal-title":"Nat. Rev. Drug Discov"},{"key":"2023041408000651700_","author":"Landrum","year":"2022"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"1560","DOI":"10.1021\/acs.jcim.0c01127","article-title":"Smiles pair encoding: a data-driven substructure tokenization algorithm for deep learning","volume":"61","author":"Li","year":"2021","journal-title":"J. Chem. Inf. Model"},{"key":"2023041408000651700_","author":"Li","year":"2020"},{"key":"2023041408000651700_","article-title":"Variational transformers for diverse response generation","author":"Lin","year":"2020","journal-title":"arXiv"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41467-021-23415-2","article-title":"Masked graph modeling for molecule generation","volume":"12","author":"Mahmood","year":"2021","journal-title":"Nat. Commun"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13321-021-00522-2","article-title":"GNINA 1.0: molecular docking with deep learning","volume":"13","author":"McNutt","year":"2021","journal-title":"J. Cheminform"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"D412","DOI":"10.1093\/nar\/gkaa913","article-title":"Pfam: the protein families database in 2021","volume":"49","author":"Mistry","year":"2021","journal-title":"Nucleic Acids Res"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1038\/s42256-020-0160-y","article-title":"Generative molecular design in low data regimes","volume":"2","author":"Moret","year":"2020","journal-title":"Nat. Mach. Intell"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"2000212","DOI":"10.1002\/minf.202000212","article-title":"Chemboost: a chemical language based approach for protein\u2013ligand binding affinity prediction","volume":"40","author":"\u00d6z\u00e7elik","year":"2021","journal-title":"Mol. Inform"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41598-017-04264-w","article-title":"Predicting the reliability of drug\u2013target interaction predictions with maximum coverage of target space","volume":"7","author":"Pe\u00f3n","year":"2017","journal-title":"Sci. Rep"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"675","DOI":"10.1007\/s10822-013-9672-4","article-title":"Estimation of the size of drug-like chemical space based on gdb-17 data","volume":"27","author":"Polishchuk","year":"2013","journal-title":"J. Comput. Aided Mol. Des"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"565644","DOI":"10.3389\/fphar.2020.565644","article-title":"Molecular sets (MOSES): a benchmarking platform for molecular generation models","volume":"11","author":"Polykovskiy","year":"2020","journal-title":"Front. Pharmacol"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"1736","DOI":"10.1021\/acs.jcim.8b00234","article-title":"Fr\u00e9chet ChemNet distance: a metric for generative models for molecules in drug discovery","volume":"58","author":"Preuer","year":"2018","journal-title":"J. Chem. Inf. Model"},{"key":"2023041408000651700_","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel","year":"2020","journal-title":"J. Mach. Learn. Res"},{"key":"2023041408000651700_","first-page":"32","author":"Renz","year":"2020"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"276","DOI":"10.1016\/S0168-9525(00)02024-2","article-title":"Emboss: the European molecular biology open software suite","volume":"16","author":"Rice","year":"2000","journal-title":"Trends Genet"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"e2016239118","DOI":"10.1073\/pnas.2016239118","article-title":"Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences","volume":"118","author":"Rives","year":"2021","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"264","DOI":"10.1162\/tacl_a_00313","article-title":"Leveraging pre-trained checkpoints for sequence generation tasks","volume":"8","author":"Rothe","year":"2020","journal-title":"Trans. Assoc. Comput. Linguist"},{"key":"2023041408000651700_","first-page":"1715","author":"Sennrich","year":"2016"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"4282","DOI":"10.1021\/acs.molpharmaceut.9b00634","article-title":"From target to drug: generative modeling for the multimodal structure-based ligand design","volume":"16","author":"Skalic","year":"2019","journal-title":"Mol. Pharm"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"2324","DOI":"10.1021\/acs.jcim.5b00559","article-title":"Zinc 15\u2013ligand discovery for everyone","volume":"55","author":"Sterling","year":"2015","journal-title":"J. Chem. Inf. Model"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"D607","DOI":"10.1093\/nar\/gky1131","article-title":"STRING v11: protein\u2013protein association networks with increased coverage, supporting functional discovery in genome-wide experimental datasets","volume":"47","author":"Szklarczyk","year":"2019","journal-title":"Nucleic Acids Res"},{"key":"2023041408000651700_","first-page":"5999","article-title":"Attention is all you need","volume":"2017","author":"Vaswani","year":"2017","journal-title":"Adv. Neural Inform. Process. Syst"},{"key":"2023041408000651700_","first-page":"31","article-title":"Smiles, a chemical language and information system. 1. Introduction to methodology and encoding rules","volume":"28","author":"Weininger","year":"1988","journal-title":"J. Chem. Inf. Model"},{"key":"2023041408000651700_","first-page":"38","volume-title":"Transformers: State-of-the-Art Natural Language Processing","author":"Wolf","year":"2020"},{"key":"2023041408000651700_","doi-asserted-by":"crossref","first-page":"7598","DOI":"10.1038\/s41598-018-25440-6","article-title":"Linguistic measures of chemical diversity and the \u201ckeywords\u201d of molecular collections","volume":"8","author":"Wo\u017aniak","year":"2018","journal-title":"Sci. Rep"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/38\/Supplement_2\/ii155\/49885882\/btac482.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/38\/Supplement_2\/ii155\/49885882\/btac482.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,26]],"date-time":"2023-11-26T04:06:19Z","timestamp":1700971579000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/38\/Supplement_2\/ii155\/6702010"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,1]]},"references-count":49,"journal-issue":{"issue":"Supplement_2","published-print":{"date-parts":[[2022,9,16]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btac482","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,9,1]]},"published":{"date-parts":[[2022,9,1]]}}}