{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,2]],"date-time":"2026-04-02T01:36:06Z","timestamp":1775093766883,"version":"3.50.1"},"reference-count":36,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,3,9]],"date-time":"2021-03-09T00:00:00Z","timestamp":1615248000000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2021,3,9]],"date-time":"2021-03-09T00:00:00Z","timestamp":1615248000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001871","name":"Funda\u00e7\u00e3o para a Ci\u00eancia e a Tecnologia","doi-asserted-by":"publisher","award":["CENTRO-01-0145-FEDER-029266"],"award-info":[{"award-number":["CENTRO-01-0145-FEDER-029266"]}],"id":[{"id":"10.13039\/501100001871","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001871","name":"Funda\u00e7\u00e3o para a Ci\u00eancia e a Tecnologia","doi-asserted-by":"publisher","award":["CENTRO-01-0145-FEDER-029266"],"award-info":[{"award-number":["CENTRO-01-0145-FEDER-029266"]}],"id":[{"id":"10.13039\/501100001871","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Cheminform"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    In this work, we explore the potential of deep learning to streamline the process of identifying new potential drugs through the computational generation of molecules with interesting biological properties. Two deep neural networks compose our targeted generation framework:\n                    <jats:italic>the Generator<\/jats:italic>\n                    , which is trained to learn the building rules of valid molecules employing SMILES strings notation, and\n                    <jats:italic>the Predictor<\/jats:italic>\n                    which evaluates the newly generated compounds by predicting their affinity for the desired target. Then, the Generator is optimized through Reinforcement Learning to produce molecules with bespoken properties. The innovation of this approach is the exploratory strategy applied during the reinforcement training process that seeks to add novelty to the generated compounds. This training strategy employs two Generators interchangeably to sample new SMILES: the initially trained model that will remain fixed and a copy of the previous one that will be updated during the training to uncover the most promising molecules. The evolution of the reward assigned by the Predictor determines how often each one is employed to select the next token of the molecule. This strategy establishes a compromise between the need to acquire more information about the chemical space and the need to sample new molecules, with the experience gained so far. To demonstrate the effectiveness of the method, the\n                    <jats:italic>Generator<\/jats:italic>\n                    is trained to design molecules with an optimized coefficient of partition and also high inhibitory power against the Adenosine\n                    <jats:inline-formula>\n                      <jats:alternatives>\n                        <jats:tex-math>$$A_{2A}$$<\/jats:tex-math>\n                        <mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                          <mml:msub>\n                            <mml:mi>A<\/mml:mi>\n                            <mml:mrow>\n                              <mml:mn>2<\/mml:mn>\n                              <mml:mi>A<\/mml:mi>\n                            <\/mml:mrow>\n                          <\/mml:msub>\n                        <\/mml:math>\n                      <\/jats:alternatives>\n                    <\/jats:inline-formula>\n                    and\n                    <jats:inline-formula>\n                      <jats:alternatives>\n                        <jats:tex-math>$$\\kappa$$<\/jats:tex-math>\n                        <mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                          <mml:mi>\u03ba<\/mml:mi>\n                        <\/mml:math>\n                      <\/jats:alternatives>\n                    <\/jats:inline-formula>\n                    opioid receptors. The results reveal that the model can effectively adjust the newly generated molecules towards the wanted direction. More importantly, it was possible to find promising sets of unique and diverse molecules, which was the main purpose of the newly implemented strategy.\n                  <\/jats:p>","DOI":"10.1186\/s13321-021-00498-z","type":"journal-article","created":{"date-parts":[[2021,3,9]],"date-time":"2021-03-09T08:02:57Z","timestamp":1615276977000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":73,"title":["Diversity oriented Deep Reinforcement Learning for targeted molecule generation"],"prefix":"10.1186","volume":"13","author":[{"given":"Tiago","family":"Pereira","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9011-0734","authenticated-orcid":false,"given":"Maryam","family":"Abbasi","sequence":"additional","affiliation":[]},{"given":"Bernardete","family":"Ribeiro","sequence":"additional","affiliation":[]},{"given":"Joel P.","family":"Arrais","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,3,9]]},"reference":[{"issue":"5","key":"498_CR1","doi-asserted-by":"publisher","first-page":"1878","DOI":"10.1093\/bib\/bby061","volume":"20","author":"AS Rifaioglu","year":"2019","unstructured":"Rifaioglu AS, Atas H, Martin MJ, Cetin-Atalay R, Atalay V, Do\u011fan T (2019) Recent applications of deep learning and machine intelligence on in silico drug discovery: methods, tools and databases. Briefings Bioinform 20(5):1878\u20131912. https:\/\/doi.org\/10.1093\/bib\/bby061","journal-title":"Briefings Bioinform"},{"issue":"1","key":"498_CR2","doi-asserted-by":"publisher","first-page":"120","DOI":"10.1021\/acscentsci.7b00512","volume":"4","author":"MHS Segler","year":"2018","unstructured":"Segler MHS, Kogej T, Tyrchan C, Waller MP (2018) Generating focused molecule libraries for drug discovery with recurrent neural networks. ACS Central Sci 4(1):120\u2013131. https:\/\/doi.org\/10.1021\/acscentsci.7b00512","journal-title":"ACS Central Sci"},{"issue":"7","key":"498_CR3","doi-asserted-by":"publisher","first-page":"3166","DOI":"10.1021\/acs.jcim.9b00325","volume":"59","author":"N St\u00e5hl","year":"2019","unstructured":"St\u00e5hl N, Falkman G, Karlsson A, Mathiason G, Bostr\u00f6m J (2019) Deep Reinforcement Learning for multiparameter optimization in de novo drug design. J Chem Inf Modeling 59(7):3166\u20133176. https:\/\/doi.org\/10.1021\/acs.jcim.9b00325","journal-title":"J Chem Inf Modeling"},{"issue":"10","key":"498_CR4","doi-asserted-by":"publisher","first-page":"2520","DOI":"10.3390\/molecules23102520","volume":"23","author":"G Hessler","year":"2018","unstructured":"Hessler G, Baringhaus K-H (2018) Artificial intelligence in drug design. Molecules 23(10):2520. https:\/\/doi.org\/10.3390\/molecules23102520","journal-title":"Molecules"},{"issue":"1","key":"498_CR5","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1186\/s13321-018-0287-6","volume":"10","author":"Y Li","year":"2018","unstructured":"Li Y, Zhang L, Liu Z (2018) Multi-objective de novo drug design with conditional graph generative model. Journal of Cheminformatics 10(1):33. https:\/\/doi.org\/10.1186\/s13321-018-0287-6","journal-title":"Journal of Cheminformatics"},{"key":"498_CR6","doi-asserted-by":"publisher","DOI":"10.1021\/ci6003652","author":"H Mauser","year":"2007","unstructured":"Mauser H, Stahl M (2007) Chemical fragment spaces for de novo design. J Chem Inform Modeling. https:\/\/doi.org\/10.1021\/ci6003652","journal-title":"J Chem Inform Modeling"},{"issue":"4","key":"498_CR7","doi-asserted-by":"publisher","first-page":"828","DOI":"10.1039\/C9ME00039A","volume":"4","author":"DC Elton","year":"2019","unstructured":"Elton DC, Boukouvalas Z, Fuge MD, Chung PW (2019) Deep learning for molecular design-a review of the state of the art. Mol Syst Design Eng 4(4):828\u2013849. https:\/\/doi.org\/10.1039\/C9ME00039A","journal-title":"Mol Syst Design Eng"},{"key":"498_CR8","doi-asserted-by":"publisher","DOI":"10.1021\/ci800308h","author":"CA Nicolaou","year":"2009","unstructured":"Nicolaou CA, Apostolakis J, Pattichis CS (2009) De novo drug design using multiobjective evolutionary graphs. J Chem Inform Model. https:\/\/doi.org\/10.1021\/ci800308h","journal-title":"J Chem Inform Model"},{"key":"498_CR9","doi-asserted-by":"publisher","DOI":"10.26434\/chemrxiv.5309668.v3","author":"B Sanchez-Lengeling","year":"2017","unstructured":"Sanchez-Lengeling B, Outeiral C, Guimaraes GL, Aspuru-Guzik A (2017) Optimizing distributions over molecular space. An Objective-Reinforced Generative Adversarial Network for Inverse-design Chemistry (ORGANIC). ChemRxiv. https:\/\/doi.org\/10.26434\/chemrxiv.5309668.v3","journal-title":"ChemRxiv"},{"issue":"1","key":"498_CR10","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1186\/s13321-017-0235-x","volume":"9","author":"M Olivecrona","year":"2017","unstructured":"Olivecrona M, Blaschke T, Engkvist O, Chen H (2017) Molecular de-novo design through deep reinforcement learning. J Cheminform 9(1):48. https:\/\/doi.org\/10.1186\/s13321-017-0235-x","journal-title":"J Cheminform"},{"issue":"7","key":"498_CR11","doi-asserted-by":"publisher","first-page":"7885","DOI":"10.1126\/sciadv.aap7885","volume":"4","author":"M Popova","year":"2018","unstructured":"Popova M, Isayev O, Tropsha A (2018) Deep reinforcement learning for de novo drug design. Sci Adv 4(7):7885. https:\/\/doi.org\/10.1126\/sciadv.aap7885","journal-title":"Sci Adv"},{"key":"498_CR12","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-019-47148-x","author":"Z Zhou","year":"2019","unstructured":"Zhou Z, Kearnes S, Li L, Zare RN (2019) Riley P (2019) Optimization of molecules via deep reinforcement learning. Sci Rep. https:\/\/doi.org\/10.1038\/s41598-019-47148-x","journal-title":"Sci Rep"},{"issue":"17","key":"498_CR13","doi-asserted-by":"publisher","first-page":"2807","DOI":"10.1016\/j.bmcl.2018.06.046","volume":"28","author":"J Panteleev","year":"2018","unstructured":"Panteleev J, Gao H, Jia L (2018) Recent applications of machine learning in medicinal chemistry. Bioorg Med Chem Lett 28(17):2807\u20132815. https:\/\/doi.org\/10.1016\/j.bmcl.2018.06.046","journal-title":"Bioorg Med Chem Lett"},{"issue":"9","key":"498_CR14","doi-asserted-by":"publisher","first-page":"1736","DOI":"10.1021\/acs.jcim.8b00234","volume":"58","author":"K Preuer","year":"2018","unstructured":"Preuer K, Renz P, Unterthiner T, Hochreiter S, Klambauer G (2018) Fr\u00e9chet chemnet distance: a metric for generative models for molecules in drug discovery. J Chem Inform Modeling 58(9):1736\u20131741","journal-title":"J Chem Inform Modeling"},{"key":"498_CR15","doi-asserted-by":"crossref","unstructured":"Benhenda M (2017) ChemGAN challenge for drug discovery: can AI reproduce natural chemical diversity? (2017). arXiv:1708.08227","DOI":"10.1101\/292177"},{"issue":"3","key":"498_CR16","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1016\/S1359-6446(97)01138-0","volume":"3","author":"J Li","year":"1998","unstructured":"Li J, Murray CW, Waszkowycz B, Young SC (1998) Targeted molecular diversity in drug discovery: Integration of structure-based design and combinatorial chemistry. Drug Discovery Today 3(3):105\u2013112. https:\/\/doi.org\/10.1016\/S1359-6446(97)01138-0","journal-title":"Drug Discovery Today"},{"issue":"1","key":"498_CR17","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1186\/s13321-019-0355-6","volume":"11","author":"X Liu","year":"2019","unstructured":"Liu X, Ye K, van Vlijmen HWT, IJzerman AP, van Westen GJP (2019) An exploration strategy improves the diversity of de novo ligands using deep reinforcement learning: a case for the adenosine A2A receptor. J Cheminform 11(1):35. https:\/\/doi.org\/10.1186\/s13321-019-0355-6","journal-title":"J Cheminform"},{"issue":"3\u20134","key":"498_CR18","doi-asserted-by":"publisher","first-page":"229","DOI":"10.1007\/BF00992696","volume":"8","author":"RJ Williams","year":"1992","unstructured":"Williams RJ (1992) Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach Learning 8(3\u20134):229\u2013256. https:\/\/doi.org\/10.1007\/BF00992696","journal-title":"Mach Learning"},{"key":"498_CR19","doi-asserted-by":"publisher","DOI":"10.1038\/nrd3955","author":"JF Chen","year":"2013","unstructured":"Chen JF, Eltzschig HK, Fredholm BB (2013) Adenosine receptors as drug targets-what are the challenges? Nat Rev Drug Discov. https:\/\/doi.org\/10.1038\/nrd3955","journal-title":"Nat Rev Drug Discov"},{"key":"498_CR20","doi-asserted-by":"publisher","first-page":"206","DOI":"10.1016\/j.ejphar.2015.05.012","volume":"763","author":"Y Shang","year":"2015","unstructured":"Shang Y, Filizola M (2015) Opioid receptors: structural and mechanistic insights into pharmacology and signaling. Eur J Pharmacol 763:206\u2013213. https:\/\/doi.org\/10.1016\/j.ejphar.2015.05.012","journal-title":"Eur J Pharmacol"},{"issue":"1\u20132","key":"498_CR21","doi-asserted-by":"publisher","first-page":"1700111","DOI":"10.1002\/minf.201700111","volume":"37","author":"A Gupta","year":"2018","unstructured":"Gupta A, M\u00fcller AT, Huisman BJH, Fuchs JA, Schneider P, Schneider G (2018) Generative recurrent networks for de novo drug design. Mol Inf 37(1\u20132):1700111. https:\/\/doi.org\/10.1002\/minf.201700111","journal-title":"Mol Inf"},{"issue":"2","key":"498_CR22","doi-asserted-by":"publisher","first-page":"270","DOI":"10.1162\/neco.1989.1.2.270","volume":"1","author":"RJ Williams","year":"1989","unstructured":"Williams RJ, Zipser D (1989) A learning algorithm for continually running fully recurrent neural networks. Neural Comput 1(2):270\u2013280. https:\/\/doi.org\/10.1162\/neco.1989.1.2.270","journal-title":"Neural Comput"},{"key":"498_CR23","unstructured":"Pascanu R, Mikolov T, Bengio Y (2013) On the difficulty of training recurrent neural networks. In: International Conference on Machine Learning, pp 1310\u20131318"},{"key":"498_CR24","doi-asserted-by":"publisher","DOI":"10.1039\/d0cp00305k","author":"K Gao","year":"2020","unstructured":"Gao K, Nguyen DD, Sresht V, Mathiowetz AM, Tu M, Wei GW (2020) Are 2D fingerprints still valuable for drug discovery? Phys Chem Chem Phys. https:\/\/doi.org\/10.1039\/d0cp00305k","journal-title":"Phys Chem Chem Phys"},{"issue":"5","key":"498_CR25","doi-asserted-by":"publisher","first-page":"742","DOI":"10.1021\/ci100050t","volume":"50","author":"D Rogers","year":"2010","unstructured":"Rogers D, Hahn M (2010) Extended-connectivity fingerprints. J Chem Inform Modeling 50(5):742\u2013754. https:\/\/doi.org\/10.1021\/ci100050t","journal-title":"J Chem Inform Modeling"},{"key":"498_CR26","doi-asserted-by":"publisher","DOI":"10.3389\/frai.2019.00017","author":"SK Chakravarti","year":"2019","unstructured":"Chakravarti SK, Alla SRM (2019) Descriptor free QSAR modeling using deep learning with long short-term memory neural networks. Front Artif Intelligence. https:\/\/doi.org\/10.3389\/frai.2019.00017","journal-title":"Front Artif Intelligence"},{"issue":"5","key":"498_CR27","doi-asserted-by":"publisher","first-page":"1054","DOI":"10.1109\/TNN.1998.712192","volume":"9","author":"RS Sutton","year":"1998","unstructured":"Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. IEEE Trans Neural Netw 9(5):1054\u20131054. https:\/\/doi.org\/10.1109\/TNN.1998.712192","journal-title":"IEEE Trans Neural Netw"},{"issue":"3\u20134","key":"498_CR28","doi-asserted-by":"publisher","first-page":"219","DOI":"10.1561\/2200000071","volume":"11","author":"V Fran\u00e7ois-Lavet","year":"2018","unstructured":"Fran\u00e7ois-Lavet V, Henderson P, Islam R, Bellemare MG, Pineau J (2018) An Introduction to Deep Reinforcement Learning. Found Trends Mach Learning 11(3\u20134):219\u2013354. https:\/\/doi.org\/10.1561\/2200000071","journal-title":"Found Trends Mach Learning"},{"key":"498_CR29","unstructured":"Landrum G (2016) Rdkit: Open-source cheminformatics software"},{"issue":"1","key":"498_CR30","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1186\/1758-2946-1-8","volume":"1","author":"P Ertl","year":"2009","unstructured":"Ertl P, Schuffenhauer A (2009) Estimation of synthetic accessibility score of drug-like molecules based on molecular complexity and fragment contributions. J Cheminform 1(1):8","journal-title":"J Cheminform"},{"key":"498_CR31","first-page":"707","volume":"10","author":"V Levenshtein","year":"1966","unstructured":"Levenshtein V (1966) Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Doklady 10:707","journal-title":"Soviet Physics Doklady"},{"issue":"13","key":"498_CR32","doi-asserted-by":"publisher","first-page":"366","DOI":"10.1093\/bioinformatics\/btn186","volume":"24","author":"Y Cao","year":"2008","unstructured":"Cao Y, Jiang T, Girke T (2008) A maximum common substructure-based algorithm for searching and predicting drug-like compounds. Bioinformatics 24(13):366\u2013374. https:\/\/doi.org\/10.1093\/bioinformatics\/btn186","journal-title":"Bioinformatics"},{"key":"498_CR33","doi-asserted-by":"publisher","DOI":"10.1021\/acs.jcim.5b00559","author":"T Sterling","year":"2015","unstructured":"Sterling T, Irwin JJ (2015) ZINC 15 - Ligand Discovery for Everyone. J Chem Inf Modeling. https:\/\/doi.org\/10.1021\/acs.jcim.5b00559","journal-title":"J Chem Inf Modeling"},{"key":"498_CR34","doi-asserted-by":"publisher","DOI":"10.1093\/nar\/gky1075","author":"D Mendez","year":"2019","unstructured":"...Mendez D, Gaulton A, Bento AP, Chambers J, De Veij M, F\u00e9lix E, Magari\u00f1os MP, Mosquera JF, Mutowo P, Nowotka M, Gordillo-Mara\u00f1\u00f3n M, Hunter F, Junco L, Mugumbate G, Rodriguez-Lopez M, Atkinson F, Bosc N, Radoux CJ, Segura-Cabrera A, Hersey A, Leach AR (2019) ChEMBL: Towards direct deposition of bioassay data. Nucleic Acids Res. https:\/\/doi.org\/10.1093\/nar\/gky1075","journal-title":"Nucleic Acids Res"},{"issue":"2","key":"498_CR35","doi-asserted-by":"publisher","first-page":"95","DOI":"10.3390\/ph12020095","volume":"12","author":"TC Beck","year":"2019","unstructured":"Beck TC, Hapstack MA, Beck KR, Dix TA (2019) Therapeutic Potential of Kappa Opioid Agonists. Pharmaceuticals 12(2):95. https:\/\/doi.org\/10.3390\/ph12020095","journal-title":"Pharmaceuticals"},{"issue":"6","key":"498_CR36","doi-asserted-by":"publisher","first-page":"1127","DOI":"10.1021\/acs.jcim.6b00088","volume":"56","author":"P Gramatica","year":"2016","unstructured":"Gramatica P, Sangion A (2016) A historical excursus on the statistical validation parameters for QSAR Models: a clarification concerning metrics and terminology. J Chem Inf Modeling 56(6):1127\u20131131. https:\/\/doi.org\/10.1021\/acs.jcim.6b00088","journal-title":"J Chem Inf Modeling"}],"container-title":["Journal of Cheminformatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s13321-021-00498-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/s13321-021-00498-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s13321-021-00498-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,9]],"date-time":"2021-03-09T08:09:55Z","timestamp":1615277395000},"score":1,"resource":{"primary":{"URL":"https:\/\/jcheminf.biomedcentral.com\/articles\/10.1186\/s13321-021-00498-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,9]]},"references-count":36,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["498"],"URL":"https:\/\/doi.org\/10.1186\/s13321-021-00498-z","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-110570\/v1","asserted-by":"object"}]},"ISSN":["1758-2946"],"issn-type":[{"value":"1758-2946","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,9]]},"assertion":[{"value":"13 November 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 February 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 March 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"21"}}