{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,31]],"date-time":"2025-12-31T06:04:20Z","timestamp":1767161060960,"version":"build-2238731810"},"reference-count":44,"publisher":"IOP Publishing","issue":"4","license":[{"start":{"date-parts":[[2020,10,8]],"date-time":"2020-10-08T00:00:00Z","timestamp":1602115200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,10,8]],"date-time":"2020-10-08T00:00:00Z","timestamp":1602115200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"DOI":"10.13039\/100000082","name":"Division of Graduate Education","doi-asserted-by":"crossref","award":["DGE-1333468"],"award-info":[{"award-number":["DGE-1333468"]}],"id":[{"id":"10.13039\/100000082","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000181","name":"Air Force Office of Scientific Research","doi-asserted-by":"crossref","award":["FA9550-18-S-0003"],"award-info":[{"award-number":["FA9550-18-S-0003"]}],"id":[{"id":"10.13039\/100000181","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Mach. Learn.: Sci. Technol."],"published-print":{"date-parts":[[2020,12,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Generative models are a sub-class of machine learning models that are capable of generating new samples with a target set of properties. In chemical and materials applications, these new samples might be drug targets, novel semiconductors, or catalysts constrained to exhibit an application-specific set of properties. Given their potential to yield high-value targets from otherwise intractable design spaces, generative models are currently under intense study with respect to how predictions can be improved through changes in model architecture and data representation. Here we explore the potential of multi-task transfer learning as a complementary approach to improving the validity and property specificity of molecules generated by such models. We have compared baseline generative models trained on a single property prediction task against models trained on additional ancillary prediction tasks and observe a generic positive impact on the validity and specificity of the multi-task models. In particular, we observe that the validity of generated structures is strongly affected by whether or not the models have chemical property data, as opposed to only syntactic structural data, supplied during learning. We demonstrate this effect in both interpolative and extrapolative scenarios (i.e., where the generative targets are poorly represented in training data) for models trained to generate high energy structures and models trained to generated structures with targeted bandgaps within certain ranges. In both instances, the inclusion of additional chemical property data improves the ability of models to generate valid, unique structures with increased property specificity. This approach requires only minor alterations to existing generative models, in many cases leveraging prediction frameworks already native to these models. Additionally, the transfer learning strategy is complementary to ongoing efforts to improve model architectures and data representation and can foreseeably be stacked on top of these developments.<\/jats:p>","DOI":"10.1088\/2632-2153\/abae75","type":"journal-article","created":{"date-parts":[[2020,8,12]],"date-time":"2020-08-12T18:30:25Z","timestamp":1597257025000},"page":"045010","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Improving the generative performance of chemical autoencoders through transfer learning"],"prefix":"10.1088","volume":"1","author":[{"given":"Nicolae C","family":"Iovanac","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7039-4039","authenticated-orcid":false,"given":"Brett M","family":"Savoie","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"266","published-online":{"date-parts":[[2020,10,8]]},"reference":[{"key":"mlstabae75bib1","article-title":"Unsupervised representation learning with deep convolutional generative adversarial networks","author":"Radford","year":"2016","journal-title":"ArXiv151106434 Cs"},{"key":"mlstabae75bib2","article-title":"Deep learning techniques for music generation \u2013 a survey","author":"Briot","year":"2019","journal-title":"ArXiv170901620 Cs"},{"key":"mlstabae75bib3","doi-asserted-by":"publisher","first-page":"360","DOI":"10.1126\/science.aat2663","article-title":"Inverse molecular design using machine learning: generative models for matter engineering","volume":"361","author":"Sanchez-Lengeling","year":"2018","journal-title":"Science"},{"key":"mlstabae75bib4","doi-asserted-by":"publisher","first-page":"445","DOI":"10.1007\/978-3-030-40245-7_21","author":"Schwalbe-Koda","year":"2020"},{"key":"mlstabae75bib5","doi-asserted-by":"publisher","first-page":"547","DOI":"10.1038\/s41586-018-0337-2","article-title":"Machine learning for molecular and materials science","volume":"559","author":"Butler","year":"2018","journal-title":"Nature"},{"key":"mlstabae75bib6","doi-asserted-by":"publisher","first-page":"24825","DOI":"10.1021\/acsami.9b01226","article-title":"Autonomous molecular design: then and now","volume":"11","author":"Dimitrov","year":"2019","journal-title":"ACS Appl. Mater. Interfaces"},{"key":"mlstabae75bib7","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41524-019-0221-0","article-title":"Recent advances and applications of machine learning in solid-state materials science","volume":"5","author":"Schmidt","year":"2019","journal-title":"Npj Comput. Mater."},{"key":"mlstabae75bib8","doi-asserted-by":"publisher","first-page":"159","DOI":"10.1016\/j.jmat.2017.08.002","article-title":"Materials discovery and design using machine learning","volume":"3","author":"Liu","year":"2017","journal-title":"J. Materiomics"},{"key":"mlstabae75bib9","doi-asserted-by":"publisher","DOI":"10.1002\/minf.201700123","article-title":"Application of generative autoencoder in de novo molecular design","volume":"37","author":"Blaschke","year":"2018","journal-title":"Mol. Inf."},{"key":"mlstabae75bib10","doi-asserted-by":"publisher","first-page":"268","DOI":"10.1021\/acscentsci.7b00572","article-title":"Automatic chemical design using a data-driven continuous representation of molecules","volume":"4","author":"G\u00f3mez-Bombarelli","year":"2018","journal-title":"ACS Cent. Sci."},{"key":"mlstabae75bib11","article-title":"Junction tree variational autoencoder for molecular graph generation","author":"Jin","year":"2019","journal-title":"ArXiv180204364 Cs Stat"},{"key":"mlstabae75bib12","doi-asserted-by":"publisher","DOI":"10.1002\/minf.201700111","article-title":"Generative recurrent networks for de novo drug design","volume":"37","author":"Gupta","year":"2018","journal-title":"Mol. Inf."},{"key":"mlstabae75bib13","doi-asserted-by":"publisher","first-page":"120","DOI":"10.1021\/acscentsci.7b00512","article-title":"Generating focused molecule libraries for drug discovery with recurrent neural networks","volume":"4","author":"Segler","year":"2018","journal-title":"ACS Cent. Sci."},{"key":"mlstabae75bib14","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1186\/s13321-019-0341-z","article-title":"Exploring the GDB-13 chemical space using deep generative models","volume":"11","author":"Ar\u00fas-Pous","year":"2019","journal-title":"J. Cheminf."},{"key":"mlstabae75bib15","article-title":"Objective-reinforced generative adversarial networks (ORGAN) for sequence generation models","author":"Guimaraes","year":"2017","journal-title":"ArXiv170510843 Cs Stat"},{"key":"mlstabae75bib16","doi-asserted-by":"publisher","DOI":"10.26434\/chemrxiv.5309668.v3","article-title":"Optimizing distributions over molecular space an objective-reinforced generative adversarial network for inverse-design chemistry (ORGANIC)","author":"Sanchez-Lengeling","year":"2017","journal-title":"figshare"},{"key":"mlstabae75bib17","article-title":"MolGAN: an implicit generative model for small molecular graphs","author":"De Cao","year":"2018","journal-title":"ArXiv180511973 Cs Stat"},{"key":"mlstabae75bib18","doi-asserted-by":"publisher","first-page":"10883","DOI":"10.18632\/oncotarget.14073","article-title":"The cornucopia of meaningful leads: applying deep adversarial autoencoders for new molecule development in oncology","volume":"8","author":"Kadurin","year":"2016","journal-title":"Oncotarget"},{"key":"mlstabae75bib19","doi-asserted-by":"publisher","first-page":"3098","DOI":"10.1021\/acs.molpharmaceut.7b00346","article-title":"DruGAN: an advanced generative adversarial autoencoder model for de novo generation of new molecules with desired molecular properties in silico","volume":"14","author":"Kadurin","year":"2017","journal-title":"Mol. Pharmaceutics"},{"key":"mlstabae75bib20","doi-asserted-by":"publisher","first-page":"4398","DOI":"10.1021\/acs.molpharmaceut.8b00839","article-title":"Entangled conditional adversarial autoencoder for de novo drug discovery","volume":"15","author":"Polykovskiy","year":"2018","journal-title":"Mol. Pharmaceutics"},{"key":"mlstabae75bib21","doi-asserted-by":"publisher","DOI":"10.1063\/1.5023563","article-title":"Machine learning-based screening of complex molecules for polymer solar cells","volume":"148","author":"J\u00f8rgensen","year":"2018","journal-title":"J. Chem. Phys."},{"key":"mlstabae75bib22","doi-asserted-by":"publisher","first-page":"1370","DOI":"10.1016\/j.matt.2019.08.017","article-title":"Inverse design of solid-state materials via a continuous representation","volume":"1","author":"Noh","year":"2019","journal-title":"Matter"},{"key":"mlstabae75bib23","article-title":"ChemGAN challenge for drug discovery: can AI reproduce natural chemical diversity?","author":"Benhenda","year":"2017","journal-title":"ArXiv170808227 Cs Stat"},{"key":"mlstabae75bib24","article-title":"Learning deep generative models of graphs","author":"Li","year":"2018","journal-title":"ArXiv180303324 Cs Stat"},{"key":"mlstabae75bib25","article-title":"GraphVAE: towards generation of small graphs using variational autoencoders","author":"Simonovsky","year":"2018","journal-title":"ArXiv180203480 Cs"},{"key":"mlstabae75bib26","doi-asserted-by":"publisher","first-page":"577","DOI":"10.1039\/C9SC04026A","article-title":"Constrained Bayesian optimization for automatic chemical design using variational autoencoders","volume":"11","author":"Griffiths","year":"2020","journal-title":"Chem. Sci."},{"key":"mlstabae75bib27","first-page":"7937-7949","article-title":"A model to search for synthesizable molecules","author":"Bradshaw","year":"2019"},{"key":"mlstabae75bib28","doi-asserted-by":"publisher","first-page":"1064","DOI":"10.1021\/acs.jpclett.8b00170","article-title":"Accelerating chemical discovery with machine learning: simulated evolution of spin crossover complexes with an artificial neural network","volume":"9","author":"Janet","year":"2018","journal-title":"J. Phys. Chem. Lett."},{"key":"mlstabae75bib29","doi-asserted-by":"publisher","first-page":"1692","DOI":"10.1039\/C8SC04175J","article-title":"Learning continuous and data-driven molecular descriptors by translating equivalent chemical representations","volume":"10","author":"Winter","year":"2019","journal-title":"Chem. Sci."},{"key":"mlstabae75bib30","doi-asserted-by":"publisher","DOI":"10.1038\/sdata.2014.22","article-title":"Quantum chemistry structures and properties of 134 kilo molecules","volume":"1","author":"Ramakrishnan","year":"2014","journal-title":"Sci. Data"},{"key":"mlstabae75bib31","doi-asserted-by":"publisher","first-page":"2324","DOI":"10.1021\/acs.jcim.5b00559","article-title":"ZINC 15 \u2013 ligand discovery for everyone","volume":"55","author":"Sterling","year":"2015","journal-title":"J. Chem. Inf. Model."},{"key":"mlstabae75bib32","doi-asserted-by":"publisher","first-page":"1345","DOI":"10.1109\/TKDE.2009.191","article-title":"Survey on transfer learning","volume":"22","author":"Pan","year":"2010","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"mlstabae75bib33","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1186\/s40537-016-0043-6","article-title":"Survey of transfer learning","volume":"3","author":"Weiss","year":"2016","journal-title":"J. Big Data"},{"key":"mlstabae75bib34","doi-asserted-by":"publisher","first-page":"5826","DOI":"10.1021\/acs.jpca.9b04195","article-title":"Accurate thermochemistry with small data sets: a bond additivity correction and transfer learning approach","volume":"123","author":"Grambow","year":"2019","journal-title":"J. Phys. Chem. A"},{"key":"mlstabae75bib35","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41467-019-10827-4","article-title":"Approaching coupled cluster accuracy with a general-purpose neural network potential through transfer learning","volume":"10","author":"Smith","year":"2019","journal-title":"Nat. Commun."},{"key":"mlstabae75bib36","doi-asserted-by":"publisher","first-page":"5291","DOI":"10.1021\/acs.jpcb.7b06896","article-title":"Transfer learning from Markov models leads to efficient sampling of related systems","volume":"122","author":"Sultan","year":"2018","journal-title":"J. Phys. Chem. B"},{"key":"mlstabae75bib37","doi-asserted-by":"publisher","first-page":"9724","DOI":"10.1021\/acs.analchem.9b01262","article-title":"MS\/MS spectrum prediction for modified peptides using pdeep2 trained by transfer learning","volume":"91","author":"Zeng","year":"2019","journal-title":"Anal. Chem."},{"key":"mlstabae75bib38","article-title":"Overcoming data scarcity with transfer learning","author":"Hutchinson","year":"2017","journal-title":"arXiv:1711.05099"},{"key":"mlstabae75bib39","doi-asserted-by":"publisher","first-page":"3679","DOI":"10.1021\/acs.jpca.0c00042","article-title":"Simpler is better: how linear prediction tasks improve transfer learning in chemical autoencoders","volume":"124","author":"Iovanac","year":"2020","journal-title":"J. Phys. Chem. A"},{"key":"mlstabae75bib40","article-title":"Massively multitask networks for drug discovery","author":"Ramsundar","year":"2015","journal-title":"arXiv:1502.02072"},{"key":"mlstabae75bib41","doi-asserted-by":"publisher","first-page":"4295","DOI":"10.1021\/acs.jpca.9b01398","article-title":"Improved chemical prediction from scarce data sets via latent space enrichment","volume":"123","author":"Iovanac","year":"2019","journal-title":"J. Phys. Chem. A"},{"key":"mlstabae75bib42","doi-asserted-by":"publisher","first-page":"513","DOI":"10.1039\/C7SC02664A","article-title":"A benchmark for molecular machine learning","volume":"9","author":"Wu","year":"2018","journal-title":"Chem. Sci."},{"key":"mlstabae75bib43","doi-asserted-by":"publisher","first-page":"1652","DOI":"10.1021\/acs.jctc.8b01176","article-title":"GFN2-XTB\u2014an accurate and broadly parametrized self-consistent tight-binding quantum chemical method with multipole electrostatics and density-dependent dispersion contributions","volume":"15","author":"Bannwarth","year":"2019","journal-title":"J. Chem. Theory Comput."},{"key":"mlstabae75bib44","doi-asserted-by":"publisher","DOI":"10.1002\/minf.201700133","article-title":"Deep generative models for molecular science","volume":"37","author":"J\u00f8rgensen","year":"2018","journal-title":"Mol. Inf."}],"updated-by":[{"DOI":"10.1088\/2632-2153\/abe194","type":"correction","label":"Correction","source":"publisher","updated":{"date-parts":[[2021,5,13]],"date-time":"2021-05-13T00:00:00Z","timestamp":1620864000000}}],"container-title":["Machine Learning: Science and Technology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abae75","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abae75\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abae75","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abae75\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abae75\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abae75","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abae75\/ampdf","content-type":"application\/pdf","content-version":"am","intended-application":"unspecified"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abae75\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abae75\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abae75\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abae75\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abae75\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abae75","content-type":"text\/html","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,11,27]],"date-time":"2021-11-27T12:05:23Z","timestamp":1638014723000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abae75"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,8]]},"references-count":44,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2020,10,8]]},"published-print":{"date-parts":[[2020,12,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2632-2153\/abae75","relation":{},"ISSN":["2632-2153"],"issn-type":[{"value":"2632-2153","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,10,8]]},"assertion":[{"value":"Improving the generative performance of chemical autoencoders through transfer learning","name":"article_title","label":"Article Title"},{"value":"Machine Learning: Science and Technology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2020 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2020-05-21","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2020-08-12","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2020-10-08","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}