{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,13]],"date-time":"2026-05-13T04:52:04Z","timestamp":1778647924069,"version":"3.51.4"},"reference-count":52,"publisher":"IOP Publishing","issue":"1","license":[{"start":{"date-parts":[[2021,12,10]],"date-time":"2021-12-10T00:00:00Z","timestamp":1639094400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,12,10]],"date-time":"2021-12-10T00:00:00Z","timestamp":1639094400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"name":"Danmarks Grundforskningsfond","award":["DNRF150"],"award-info":[{"award-number":["DNRF150"]}]},{"DOI":"10.13039\/100008398","name":"Villum Fonden","doi-asserted-by":"crossref","award":["16562"],"award-info":[{"award-number":["16562"]}],"id":[{"id":"10.13039\/100008398","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Mach. Learn.: Sci. Technol."],"published-print":{"date-parts":[[2022,3,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Chemical space is routinely explored by machine learning methods to discover interesting molecules, before time-consuming experimental synthesizing is attempted. However, these methods often rely on a graph representation, ignoring 3D information necessary for determining the stability of the molecules. We propose a reinforcement learning (RL) approach for generating molecules in Cartesian coordinates allowing for quantum chemical prediction of the stability. To improve sample-efficiency we learn basic chemical rules from imitation learning (IL) on the GDB-11 database to create an initial model applicable for all stoichiometries. We then deploy multiple copies of the model conditioned on a specific stoichiometry in a RL setting. The models correctly identify low energy molecules in the database and produce novel isomers not found in the training set. Finally, we apply the model to larger molecules to show how RL further refines the IL model in domains far from the training data.<\/jats:p>","DOI":"10.1088\/2632-2153\/ac3eb4","type":"journal-article","created":{"date-parts":[[2021,12,1]],"date-time":"2021-12-01T04:48:56Z","timestamp":1638334136000},"page":"015008","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["Generating stable molecules using imitation and reinforcement learning"],"prefix":"10.1088","volume":"3","author":[{"given":"S\u00f8ren","family":"Ager Meldgaard","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jonas","family":"K\u00f6hler","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Henrik","family":"Lund Mortensen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mads-Peter V","family":"Christiansen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4169-9324","authenticated-orcid":false,"given":"Frank","family":"No\u00e9","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7849-6347","authenticated-orcid":false,"given":"Bj\u00f8rk","family":"Hammer","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"266","published-online":{"date-parts":[[2021,12,10]]},"reference":[{"key":"mlstac3eb4bib1","doi-asserted-by":"publisher","first-page":"9973","DOI":"10.1021\/j100141a013","volume":"97","author":"Hartke","year":"1993","journal-title":"J. Phys. Chem."},{"key":"mlstac3eb4bib2","doi-asserted-by":"publisher","first-page":"e11","DOI":"10.7717\/peerj-pchem.11","volume":"2","author":"Henault","year":"2020","journal-title":"PeerJ Phys. Chem."},{"key":"mlstac3eb4bib3","doi-asserted-by":"publisher","first-page":"5111","DOI":"10.1021\/jp970984n","volume":"101","author":"Wales","year":"1997","journal-title":"J. Phys. Chem. A"},{"key":"mlstac3eb4bib4","first-page":"pp 1942","article-title":"Particle swarm optimization","volume":"vol 4","author":"Kennedy","year":"1995"},{"key":"mlstac3eb4bib5","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevB.97.195424","volume":"97","author":"Kolsbjerg","year":"2018","journal-title":"Phys. Rev. B"},{"key":"mlstac3eb4bib6","doi-asserted-by":"publisher","DOI":"10.1063\/1.5048290","volume":"149","author":"Meldgaard","year":"2018","journal-title":"J. Chem. Phys."},{"key":"mlstac3eb4bib7","doi-asserted-by":"publisher","first-page":"8016","DOI":"10.1039\/C9SC01928F","volume":"10","author":"Winter","year":"2019","journal-title":"Chem. Sci."},{"key":"mlstac3eb4bib8","doi-asserted-by":"publisher","first-page":"332","DOI":"10.2174\/138620709788167980","volume":"12","author":"Melville","year":"2009","journal-title":"Comb. Chem. High Throughput Screen."},{"key":"mlstac3eb4bib9","doi-asserted-by":"publisher","first-page":"3347","DOI":"10.2174\/1381612824666180607124038","volume":"24","author":"Carpenter","year":"2018","journal-title":"Curr. Pharm. Des."},{"key":"mlstac3eb4bib10","doi-asserted-by":"publisher","first-page":"177","DOI":"10.1021\/ci049714+","volume":"45","author":"Irwin","year":"2005","journal-title":"J. Chem. Inf. Model."},{"key":"mlstac3eb4bib11","doi-asserted-by":"publisher","first-page":"2864","DOI":"10.1021\/ci300415d","volume":"52","author":"Ruddigkeit","year":"2012","journal-title":"J. Chem. Inf. Model."},{"key":"mlstac3eb4bib12","doi-asserted-by":"publisher","DOI":"10.1038\/sdata.2014.22","volume":"1","author":"Ramakrishnan","year":"2014","journal-title":"Sci. Data"},{"key":"mlstac3eb4bib13","doi-asserted-by":"publisher","DOI":"10.1088\/2053-1583\/aacfc1","volume":"5","author":"Haastrup","year":"2018","journal-title":"2D Mater."},{"key":"mlstac3eb4bib14","doi-asserted-by":"publisher","first-page":"75","DOI":"10.1038\/s41597-019-0081-y","volume":"6","author":"Winther","year":"2019","journal-title":"Sci. Data"},{"key":"mlstac3eb4bib15","doi-asserted-by":"crossref","DOI":"10.1186\/s13321-018-0287-6","article-title":"Multi-objective de novo drug design with conditional graph generative model","author":"Li","year":"2018"},{"key":"mlstac3eb4bib16","doi-asserted-by":"publisher","DOI":"10.1002\/minf.201700111","volume":"37","author":"Gupta","year":"2018","journal-title":"Mol. Inform."},{"key":"mlstac3eb4bib17","doi-asserted-by":"publisher","first-page":"268","DOI":"10.1021\/acscentsci.7b00572","volume":"4","author":"G\u00f3mez-Bombarelli","year":"2018","journal-title":"ACS Cent. Sci."},{"key":"mlstac3eb4bib18","doi-asserted-by":"publisher","first-page":"1038","DOI":"10.1038\/s41587-019-0224-x","volume":"37","author":"Zhavoronkov","year":"2019","journal-title":"Nat. Biotechnol."},{"key":"mlstac3eb4bib19","doi-asserted-by":"publisher","first-page":"eaaw1147","DOI":"10.1126\/science.aaw1147","volume":"365","author":"No\u00e9","year":"2019","journal-title":"Science"},{"key":"mlstac3eb4bib20","article-title":"Generating valid Euclidean distance matrices","author":"Hoffmann","year":"2019"},{"key":"mlstac3eb4bib21","article-title":"Symmetry-adapted generation of 3D point sets for the targeted discovery of molecules","author":"Gebauer","year":"2019"},{"key":"mlstac3eb4bib22","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-019-56773-5","volume":"9","author":"Mansimov","year":"2019","journal-title":"Sci. Rep."},{"key":"mlstac3eb4bib23","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1186\/s13321-019-0404-1","volume":"12","author":"Maziarka","year":"2020","journal-title":"J. Cheminform."},{"key":"mlstac3eb4bib24","first-page":"pp 5361","article-title":"Equivariant flows: exact likelihood generative learning for symmetric densities","author":"K\u00f6hler","year":"2020"},{"key":"mlstac3eb4bib25","article-title":"E(n) equivariant normalizing flows for molecule generation in 3D","author":"Satorras","year":"2021"},{"key":"mlstac3eb4bib26","article-title":"Permutation-invariant variational autoencoder for graph-level representation learning","author":"Winter","year":"2021"},{"key":"mlstac3eb4bib27","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1186\/s13321-017-0235-x","volume":"9","author":"Olivecrona","year":"2017","journal-title":"J. Cheminform."},{"key":"mlstac3eb4bib28","doi-asserted-by":"publisher","first-page":"1194","DOI":"10.1021\/acs.jcim.7b00690","volume":"58","author":"Putin","year":"2018","journal-title":"J. Chem. Inf. Model."},{"key":"mlstac3eb4bib29","article-title":"Molgan: An implicit generative model for small molecular graphs","author":"Cao","year":"2018"},{"key":"mlstac3eb4bib30","doi-asserted-by":"publisher","first-page":"eaa7885","DOI":"10.1126\/sciadv.aap7885","volume":"4","author":"Popova","year":"2018","journal-title":"Sci. Adv."},{"key":"mlstac3eb4bib31","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-019-47148-x","volume":"9","author":"Zhou","year":"2019","journal-title":"Sci. Rep."},{"key":"mlstac3eb4bib32","doi-asserted-by":"publisher","first-page":"970","DOI":"10.1021\/acscentsci.9b00055","volume":"5","author":"Schreck","year":"2019","journal-title":"ACS Cent. Sci."},{"key":"mlstac3eb4bib33","doi-asserted-by":"publisher","DOI":"10.1063\/1.5108871","volume":"151","author":"J\u00f8rgensen","year":"2019","journal-title":"J. Chem. Phys."},{"key":"mlstac3eb4bib34","first-page":"pp 8959","article-title":"Reinforcement learning for molecular design guided by quantum mechanics","author":"Simm","year":"2020"},{"key":"mlstac3eb4bib35","doi-asserted-by":"publisher","DOI":"10.1088\/1361-648x\/ab94f2","volume":"32","author":"Meldgaard","year":"2020","journal-title":"J. Phys.: Condens. Matter"},{"key":"mlstac3eb4bib36","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevB.102.075427","volume":"102","author":"Mortensen","year":"2020","journal-title":"Phys. Rev. B"},{"key":"mlstac3eb4bib37","doi-asserted-by":"publisher","DOI":"10.1063\/5.0015571","volume":"153","author":"Christiansen","year":"2020","journal-title":"J. Chem. Phys."},{"key":"mlstac3eb4bib38","article-title":"Symmetry-aware actor-critic for 3D molecular design","author":"Simm","year":"2021"},{"key":"mlstac3eb4bib39","doi-asserted-by":"publisher","first-page":"1504","DOI":"10.1002\/anie.200462457","volume":"44","author":"Fink","year":"2005","journal-title":"Angew. Chem., Int. Ed. Engl."},{"key":"mlstac3eb4bib40","doi-asserted-by":"publisher","first-page":"5714","DOI":"10.1021\/acs.jcim.0c00174","volume":"60","author":"Gao","year":"2020","journal-title":"J. Chem. Inf. Model."},{"key":"mlstac3eb4bib41","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1021\/ci00057a005","volume":"28","author":"Weininger","year":"1988","journal-title":"J. Chem. Inf. Comput. Sci."},{"key":"mlstac3eb4bib42","doi-asserted-by":"publisher","DOI":"10.1063\/1.5019779","volume":"148","author":"Sch\u00fctt","year":"2018","journal-title":"J. Chem. Phys."},{"key":"mlstac3eb4bib43","article-title":"Attention is all you need","volume":"vol 30","author":"Vaswani","year":"2017"},{"key":"mlstac3eb4bib44","doi-asserted-by":"publisher","first-page":"5573","DOI":"10.24963\/ijcai.2018\/787","author":"Machado","year":"2018","journal-title":"J. Artif. Intell. Res."},{"key":"mlstac3eb4bib45","article-title":"Layer normalization","author":"Ba","year":"2016"},{"key":"mlstac3eb4bib46","article-title":"RDKit: open-source cheminformatics","author":"Landrum"},{"key":"mlstac3eb4bib47","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevB.71.035109","volume":"71","author":"Mortensen","year":"2005","journal-title":"Phys. Rev. B"},{"key":"mlstac3eb4bib48","doi-asserted-by":"publisher","DOI":"10.1088\/0953-8984\/22\/25\/253202","volume":"22","author":"Enkovaara","year":"2010","journal-title":"J. Phys.: Condens. Matter"},{"key":"mlstac3eb4bib49","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevB.80.195112","volume":"80","author":"Larsen","year":"2009","journal-title":"Phys. Rev. B"},{"key":"mlstac3eb4bib50","article-title":"PyTorch: an imperative style, high-performance deep learning library","author":"Paszke","year":"2019"},{"key":"mlstac3eb4bib51","article-title":"Adam: a method for stochastic optimization","author":"Kingma","year":"2014"},{"key":"mlstac3eb4bib52","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevB.87.184115","volume":"87","author":"Bart\u00f3k","year":"2013","journal-title":"Phys. Rev. B"}],"container-title":["Machine Learning: Science and Technology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ac3eb4","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ac3eb4\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ac3eb4","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ac3eb4\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ac3eb4\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ac3eb4\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ac3eb4\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ac3eb4\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,12,15]],"date-time":"2021-12-15T13:33:03Z","timestamp":1639575183000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ac3eb4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,10]]},"references-count":52,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,12,10]]},"published-print":{"date-parts":[[2022,3,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2632-2153\/ac3eb4","relation":{},"ISSN":["2632-2153"],"issn-type":[{"value":"2632-2153","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12,10]]},"assertion":[{"value":"Generating stable molecules using imitation and reinforcement learning","name":"article_title","label":"Article Title"},{"value":"Machine Learning: Science and Technology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2021 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2021-08-27","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2021-11-30","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2021-12-10","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}