{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,24]],"date-time":"2026-03-24T18:07:29Z","timestamp":1774375649964,"version":"3.50.1"},"reference-count":44,"publisher":"Oxford University Press (OUP)","issue":"Supplement_1","license":[{"start":{"date-parts":[[2021,7,12]],"date-time":"2021-07-12T00:00:00Z","timestamp":1626048000000},"content-version":"vor","delay-in-days":11,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Portuguese Research Agency FCT"},{"name":"D4 \u2013 Deep Drug Discovery and Deployment","award":["CENTRO-01-0145-FEDER-029266"],"award-info":[{"award-number":["CENTRO-01-0145-FEDER-029266"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,8,4]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Motivation<\/jats:title>\n                  <jats:p>The process of placing new drugs into the market is time-consuming, expensive and complex. The application of computational methods for designing molecules with bespoke properties can contribute to saving resources throughout this process. However, the fundamental properties to be optimized are often not considered or conflicting with each other. In this work, we propose a novel approach to consider both the biological property and the bioavailability of compounds through a deep reinforcement learning framework for the targeted generation of compounds. We aim to obtain a promising set of selective compounds for the adenosine A2A receptor and, simultaneously, that have the necessary properties in terms of solubility and permeability across the blood\u2013brain barrier to reach the site of action. The cornerstone of the framework is based on a recurrent neural network architecture, the Generator. It seeks to learn the building rules of valid molecules to sample new compounds further. Also, two Predictors are trained to estimate the properties of interest of the new molecules. Finally, the fine-tuning of the Generator was performed with reinforcement learning, integrated with multi-objective optimization and exploratory techniques to ensure that the Generator is adequately biased.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>The biased Generator can generate an interesting set of molecules, with approximately 85% having the two fundamental properties biased as desired. Thus, this approach has transformed a general molecule generator into a model focused on optimizing specific objectives. Furthermore, the molecules\u2019 synthesizability and drug-likeness demonstrate the potential applicability of the de novo drug design in medicinal chemistry.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Availability and implementation<\/jats:title>\n                  <jats:p>All code is publicly available in the https:\/\/github.com\/larngroup\/De-Novo-Drug-Design.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Supplementary information<\/jats:title>\n                  <jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btab301","type":"journal-article","created":{"date-parts":[[2021,6,7]],"date-time":"2021-06-07T20:02:41Z","timestamp":1623096161000},"page":"i84-i92","source":"Crossref","is-referenced-by-count":23,"title":["Optimizing blood\u2013brain barrier permeation through deep reinforcement learning for <i>de novo<\/i> drug design"],"prefix":"10.1093","volume":"37","author":[{"given":"Tiago","family":"Pereira","sequence":"first","affiliation":[{"name":"CSUC\/DEI, University of Coimbra , Coimbra 3030-290, Portugal"},{"name":"IEETA\/DETI, University of Aveiro , Aveiro 3810-193, Portugal"}]},{"given":"Maryam","family":"Abbasi","sequence":"additional","affiliation":[{"name":"CSUC\/DEI, University of Coimbra , Coimbra 3030-290, Portugal"}]},{"given":"Jos\u00e9 Luis","family":"Oliveira","sequence":"additional","affiliation":[{"name":"IEETA\/DETI, University of Aveiro , Aveiro 3810-193, Portugal"}]},{"given":"Bernardete","family":"Ribeiro","sequence":"additional","affiliation":[{"name":"CSUC\/DEI, University of Coimbra , Coimbra 3030-290, Portugal"}]},{"given":"Joel","family":"Arrais","sequence":"additional","affiliation":[{"name":"CSUC\/DEI, University of Coimbra , Coimbra 3030-290, Portugal"}]}],"member":"286","published-online":{"date-parts":[[2021,7,12]]},"reference":[{"key":"2023062410292970000_btab301-B1","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1021\/ci034205d","article-title":"Blood\u2013brain barrier permeation models: discriminating between potential CNS and non-CNS drugs including P-glycoprotein substrates","volume":"44","author":"Adenot","year":"2004","journal-title":"J. Chem. Inf. Comput. Sci"},{"key":"2023062410292970000_btab301-B2","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1007\/s00018-015-2050-8","article-title":"Factors controlling permeability of the blood\u2013brain barrier","volume":"73","author":"Almutairi","year":"2016","journal-title":"Cell. Mol. Life Sci"},{"key":"2023062410292970000_btab301-B3","doi-asserted-by":"crossref","first-page":"107377","DOI":"10.1016\/j.compbiolchem.2020.107377","article-title":"A recurrent neural network model to predict blood\u2013brain barrier permeability","volume":"89","author":"Alsenan","year":"2020","journal-title":"Comput. Biol. Chem"},{"key":"2023062410292970000_btab301-B4","author":""},{"key":"2023062410292970000_btab301-B5","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1038\/nchem.1243","article-title":"Quantifying the chemical beauty of drugs","volume":"4","author":"Bickerton","year":"2012","journal-title":"Nat. Chem"},{"key":"2023062410292970000_btab301-B6","author":"Brys","year":"2013"},{"key":"2023062410292970000_btab301-B7","doi-asserted-by":"crossref","first-page":"17","DOI":"10.3389\/frai.2019.00017","article-title":"Descriptor free QSAR modeling using deep learning with long short-term memory neural networks","volume":"2","author":"Chakravarti","year":"2019","journal-title":"Front. Artif. Intell"},{"key":"2023062410292970000_btab301-B8","doi-asserted-by":"crossref","first-page":"592","DOI":"10.1016\/j.tips.2019.06.004","article-title":"Advancing drug discovery via artificial intelligence","volume":"40","author":"Chan","year":"2019","journal-title":"Trends Pharmacol. Sci"},{"key":"2023062410292970000_btab301-B9","doi-asserted-by":"crossref","first-page":"1241","DOI":"10.1016\/j.drudis.2018.01.039","article-title":"The rise of deep learning in drug discovery","volume":"23","author":"Chen","year":"2018","journal-title":"Drug Discov. Today"},{"key":"2023062410292970000_btab301-B10","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1038\/nrd3955","article-title":"Adenosine receptors as drug targets-what are the challenges?","volume":"12","author":"Chen","year":"2013","journal-title":"Nat. Rev. Drug Discov"},{"key":"2023062410292970000_btab301-B11","author":"Deng","year":"2020"},{"key":"2023062410292970000_btab301-B12","doi-asserted-by":"crossref","first-page":"543","DOI":"10.1016\/j.asoc.2014.09.042","article-title":"Evolutionary algorithms for de novo drug design \u2013 a survey","volume":"27","author":"Devi","year":"2015","journal-title":"Appl. Soft Comput"},{"key":"2023062410292970000_btab301-B13","author":"Duvenaud","year":"2015"},{"key":"2023062410292970000_btab301-B14","doi-asserted-by":"publisher","author":"Ehrgott","year":"2005","DOI":"10.1007\/978-3-662-22199-0"},{"key":"2023062410292970000_btab301-B15","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1186\/1758-2946-1-8","article-title":"Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions","volume":"1","author":"Ertl","year":"2009","journal-title":"J. Cheminf"},{"key":"2023062410292970000_btab301-B16","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1561\/2200000071","article-title":"An introduction to deep reinforcement learning","volume":"11","author":"Fran\u00e7ois-Lavet","year":"2018","journal-title":"Found. Trends Mach. Learn"},{"key":"2023062410292970000_btab301-B17","doi-asserted-by":"crossref","first-page":"1127","DOI":"10.1021\/acs.jcim.6b00088","article-title":"A historical excursus on the statistical validation parameters for QSAR models: a clarification concerning metrics and terminology","volume":"56","author":"Gramatica","year":"2016","journal-title":"J. Chem. Inf. Model"},{"key":"2023062410292970000_btab301-B18","doi-asserted-by":"crossref","first-page":"1700111","DOI":"10.1002\/minf.201700111","article-title":"Generative recurrent networks for de novo drug design","volume":"37","author":"Gupta","year":"2018","journal-title":"Mol. Inf"},{"key":"2023062410292970000_btab301-B19","doi-asserted-by":"crossref","first-page":"1239","DOI":"10.1111\/j.1476-5381.2010.01127.x","article-title":"Principles of early drug discovery","volume":"162","author":"Hughes","year":"2011","journal-title":"Br. J. Pharmacol"},{"key":"2023062410292970000_btab301-B20","doi-asserted-by":"publisher","DOI":"10.1016\/C2013-0-18378-X","volume-title":"Drug-like Properties: Concepts, Structure Design and Methods","author":"Kerns","year":"2008"},{"key":"2023062410292970000_btab301-B21","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1186\/s13321-020-00454-3","article-title":"DeepGraphMolGen, a multi-objective, computational strategy for generating molecules with desirable properties: a graph convolution and reinforcement learning approach","volume":"12","author":"Khemchandani","year":"2020","journal-title":"J. Cheminf"},{"key":"2023062410292970000_btab301-B22","author":"Landrum","year":"2019"},{"key":"2023062410292970000_btab301-B23","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1186\/s13321-019-0355-6","article-title":"An exploration strategy improves the diversity of de novo ligands using deep reinforcement learning: a case for the adenosine A2A receptor","volume":"11","author":"Liu","year":"2019","journal-title":"J. Cheminf"},{"key":"2023062410292970000_btab301-B24","doi-asserted-by":"crossref","first-page":"D930","DOI":"10.1093\/nar\/gky1075","article-title":"ChEMBL: towards direct deposition of bioassay data","volume":"47","author":"Mendez","year":"2019","journal-title":"Nucleic Acids Res"},{"key":"2023062410292970000_btab301-B25","doi-asserted-by":"publisher","author":"Nguyen","year":"2018","DOI":"10.1016\/j.engappai.2020.103915"},{"key":"2023062410292970000_btab301-B26","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1021\/ci800308h","article-title":"De novo drug design using multiobjective evolutionary graphs","volume":"49","author":"Nicolaou","year":"2009","journal-title":"J. Chem. Inf. Model"},{"key":"2023062410292970000_btab301-B27","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1186\/s13321-017-0235-x","article-title":"Molecular de-novo design through deep reinforcement learning","volume":"9","author":"Olivecrona","year":"2017","journal-title":"J. Cheminf"},{"key":"2023062410292970000_btab301-B28","author":"Pascanu","year":"2013"},{"key":"2023062410292970000_btab301-B29","doi-asserted-by":"crossref","first-page":"eaap7885","DOI":"10.1126\/sciadv.aap7885","article-title":"Deep reinforcement learning for de novo drug design","volume":"4","author":"Popova","year":"2018","journal-title":"Sci. Adv"},{"key":"2023062410292970000_btab301-B30","doi-asserted-by":"crossref","first-page":"742","DOI":"10.1021\/ci100050t","article-title":"Extended-connectivity fingerprints","volume":"50","author":"Rogers","year":"2010","journal-title":"J. Chem. Inf. Model"},{"key":"2023062410292970000_btab301-B31","doi-asserted-by":"crossref","first-page":"16774","DOI":"10.1021\/acsomega.9b01512","article-title":"To pass or not to pass: predicting the blood\u2013brain barrier permeability with the 3D-RISM-KH molecular solvation theory","volume":"4","author":"Roy","year":"2019","journal-title":"ACS Omega"},{"key":"2023062410292970000_btab301-B32","author":"Ruder","year":"2016"},{"key":"2023062410292970000_btab301-B33","doi-asserted-by":"crossref","first-page":"360","DOI":"10.1126\/science.aat2663","article-title":"Inverse molecular design using machine learning: generative models for matter engineering","volume":"361","author":"Sanchez-Lengeling","year":"2018","journal-title":"Science"},{"key":"2023062410292970000_btab301-B34","article-title":"Optimizing distributions over molecular space. An objective-reinforced generative adversarial network for inverse-design chemistry (ORGANIC)","author":"Sanchez-Lengeling","year":"2017","journal-title":"10.26434\/chemrxiv.5309668.v3."},{"key":"2023062410292970000_btab301-B35","doi-asserted-by":"crossref","first-page":"3166","DOI":"10.1021\/acs.jcim.9b00325","article-title":"Deep reinforcement learning for multiparameter optimization in de novo drug design","volume":"59","author":"St\u00e5hl","year":"2019","journal-title":"J. Chem. Inf. Model"},{"key":"2023062410292970000_btab301-B36","doi-asserted-by":"crossref","first-page":"2324","DOI":"10.1021\/acs.jcim.5b00559","article-title":"ZINC 15 \u2013 ligand discovery for everyone","volume":"55","author":"Sterling","year":"2015","journal-title":"J. Chem. Inf. Model"},{"key":"2023062410292970000_btab301-B37","doi-asserted-by":"crossref","first-page":"1054","DOI":"10.1109\/TNN.1998.712192","article-title":"Reinforcement learning: an introduction","volume":"9","author":"Sutton","year":"1998","journal-title":"IEEE Transac. Neural Netw"},{"key":"2023062410292970000_btab301-B38","doi-asserted-by":"publisher","first-page":"191","DOI":"10.1109\/ADPRL.2013.6615007","author":"Van Moffaert","year":"2013"},{"key":"2023062410292970000_btab301-B39","doi-asserted-by":"crossref","first-page":"2189","DOI":"10.1002\/cmdc.201800533","article-title":"In silico prediction of blood\u2013brain barrier permeability of compounds by machine learning and resampling methods","volume":"13","author":"Wang","year":"2018","journal-title":"ChemMedChem"},{"key":"2023062410292970000_btab301-B40","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1007\/BF00992696","article-title":"Simple statistical gradient-following algorithms for connectionist reinforcement learning","volume":"8","author":"Williams","year":"1992","journal-title":"Mach. Learn"},{"key":"2023062410292970000_btab301-B41","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1039\/C7SC02664A","article-title":"Moleculenet: a benchmark for molecular machine learning","volume":"9","author":"Wu","year":"2018","journal-title":"Chem. Sci"},{"key":"2023062410292970000_btab301-B42","volume-title":"Advances in Neural Information Processing Systems","author":"You","year":"2018"},{"key":"2023062410292970000_btab301-B43","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1021\/ci600312d","article-title":"Predicting penetration across the blood\u2013brain barrier from simple descriptors and fragmentation schemes","volume":"47","author":"Zhao","year":"2007","journal-title":"J. Chem. Inf. Model"},{"key":"2023062410292970000_btab301-B44","doi-asserted-by":"crossref","first-page":"10752","DOI":"10.1038\/s41598-019-47148-x","article-title":"Optimization of molecules via deep reinforcement learning","volume":"9","author":"Zhou","year":"2019","journal-title":"Sci. Rep"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/37\/Supplement_1\/i84\/50694459\/btab301.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/37\/Supplement_1\/i84\/50694459\/btab301.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,25]],"date-time":"2023-06-25T00:21:44Z","timestamp":1687652504000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/37\/Supplement_1\/i84\/6319684"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,1]]},"references-count":44,"journal-issue":{"issue":"Supplement_1","published-print":{"date-parts":[[2021,8,4]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btab301","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,7,1]]},"published":{"date-parts":[[2021,7,1]]}}}