{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,22]],"date-time":"2026-01-22T18:12:03Z","timestamp":1769105523967,"version":"3.49.0"},"reference-count":69,"publisher":"IOP Publishing","issue":"1","license":[{"start":{"date-parts":[[2024,1,19]],"date-time":"2024-01-19T00:00:00Z","timestamp":1705622400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,1,19]],"date-time":"2024-01-19T00:00:00Z","timestamp":1705622400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"name":"US Department of Energy, Office of Science"},{"name":"Center for Nanophase Materials Sciences"},{"DOI":"10.13039\/100017535","name":"Energy Frontier Research Centers","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100017535","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100006132","name":"Office of Science","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100006132","id-type":"DOI","asserted-by":"crossref"}]},{"name":"The Center for the Science of Synthesis Across Scales","award":["DE-SC0019288"],"award-info":[{"award-number":["DE-SC0019288"]}]},{"name":"University of Washington"},{"name":"Sciences and Engineering Division of the Basic Energy Sciences"},{"DOI":"10.13039\/100006228","name":"Oak Ridge National Laboratory","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100006228","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000015","name":"US Department of Energy","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100000015","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Mach. Learn.: Sci. Technol."],"published-print":{"date-parts":[[2024,3,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>The ability of deep learning methods to perform classification and regression tasks relies heavily on their capacity to uncover manifolds in high-dimensional data spaces and project them into low-dimensional representation spaces. In this study, we investigate the structure and character of the manifolds generated by classical variational autoencoder (VAE) approaches and deep kernel learning (DKL). In the former case, the structure of the latent space is determined by the properties of the input data alone, while in the latter, the latent manifold forms as a result of an active learning process that balances the data distribution and target functionalities. We show that DKL with active learning can produce a more compact and smooth latent space which is more conducive to optimization compared to previously reported methods, such as the VAE. We demonstrate this behavior using a simple cards dataset and extend it to the optimization of domain-generated trajectories in physical systems. Our findings suggest that latent manifolds constructed through active learning have a more beneficial structure for optimization problems, especially in feature-rich target-poor scenarios that are common in domain sciences, such as materials synthesis, energy storage, and molecular discovery. The Jupyter Notebooks that encapsulate the complete analysis accompany the article.<\/jats:p>","DOI":"10.1088\/2632-2153\/ad1a4f","type":"journal-article","created":{"date-parts":[[2024,1,3]],"date-time":"2024-01-03T22:43:19Z","timestamp":1704321799000},"page":"015012","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["Deep kernel methods learn better: from cards to process optimization"],"prefix":"10.1088","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8022-647X","authenticated-orcid":true,"given":"Mani","family":"Valleti","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4692-8579","authenticated-orcid":true,"given":"Rama K","family":"Vasudevan","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2570-4592","authenticated-orcid":true,"given":"Maxim A","family":"Ziatdinov","sequence":"additional","affiliation":[]},{"given":"Sergei V","family":"Kalinin","sequence":"additional","affiliation":[]}],"member":"266","published-online":{"date-parts":[[2024,1,19]]},"reference":[{"key":"mlstad1a4fbib1","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1038\/s41524-021-00569-7","article-title":"Ensemble learning-iterative training machine learning for uncertainty quantification and automated experiment in atom-resolved microscopy","volume":"7","author":"Ghosh","year":"2021","journal-title":"npj Comput. Mater."},{"key":"mlstad1a4fbib2","doi-asserted-by":"publisher","first-page":"1451","DOI":"10.1109\/WACV.2018.00163","article-title":"Understanding Convolution for Semantic Segmentation","author":"Wang","year":"2018"},{"key":"mlstad1a4fbib3","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1007\/s11263-014-0777-6","article-title":"Indoor scene understanding with RGB-D images: bottom-up segmentation, object detection and semantic segmentation","volume":"112","author":"Gupta","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"mlstad1a4fbib4","doi-asserted-by":"publisher","author":"Lewis-Beck","year":"2016","DOI":"10.4135\/9781483396774"},{"key":"mlstad1a4fbib5","doi-asserted-by":"publisher","first-page":"533","DOI":"10.1007\/s11750-022-00631-7","article-title":"Regression markets and application to energy forecasting","volume":"30","author":"Pinson","year":"2022","journal-title":"Top"},{"key":"mlstad1a4fbib6","first-page":"121","article-title":"Classification, application, and creation of landscape indices","volume":"13","author":"Chen","year":"2002","journal-title":"Ying Yong Sheng Tai Xue Bao= J. Appl. Ecol."},{"key":"mlstad1a4fbib7","doi-asserted-by":"publisher","DOI":"10.20944\/preprints201811.0601.v1","article-title":"Unmanned aerial vehicle classification, applications and challenges: a review Preprints","author":"Singhal","year":"2018"},{"key":"mlstad1a4fbib8","doi-asserted-by":"publisher","first-page":"721","DOI":"10.1016\/j.knosys.2008.03.026","article-title":"An application of supervised and unsupervised learning approaches to telecommunications fraud detection","volume":"21","author":"Hilas","year":"2008","journal-title":"Knowl.-Based Syst."},{"key":"mlstad1a4fbib9","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-24211-8","volume":"vol 9","author":"Celebi","year":"2016","edition":"1 edn"},{"key":"mlstad1a4fbib10","doi-asserted-by":"publisher","first-page":"1858","DOI":"10.1109\/LRA.2023.3243474","article-title":"Discriminative active learning for robotic grasping in cluttered scene","volume":"8","author":"Wei","year":"2023","journal-title":"IEEE Robot. Autom. Lett."},{"key":"mlstad1a4fbib11","doi-asserted-by":"publisher","first-page":"577","DOI":"10.1007\/s00466-023-02279-x","article-title":"A thermodynamics-informed active learning approach to perception and reasoning about fluids","volume":"72","author":"Moya","year":"2023","journal-title":"Comput. Mech."},{"key":"mlstad1a4fbib12","article-title":"Review of control principles for active learning in robotics","author":"Behera"},{"key":"mlstad1a4fbib13","doi-asserted-by":"publisher","first-page":"325","DOI":"10.1109\/MASS56207.2022.00054","article-title":"Time constraint finite-horizon path planning solution for micromouse extreme problem","author":"Bose","year":"2022"},{"key":"mlstad1a4fbib14","doi-asserted-by":"publisher","first-page":"2816","DOI":"10.1109\/ITSC48978.2021.9564689","article-title":"Weather and light level classification for autonomous driving: dataset, baseline and active learning","author":"Dhananjaya","year":"2021"},{"key":"mlstad1a4fbib15","article-title":"Exploring diversity-based active learning for 3D object detection in autonomous driving","author":"Liang","year":"2022"},{"key":"mlstad1a4fbib16","doi-asserted-by":"publisher","first-page":"667","DOI":"10.1109\/IVS.2019.8814236","article-title":"Deep active learning for efficient training of a LiDAR 3D object detector","author":"Feng","year":"2019"},{"key":"mlstad1a4fbib17","doi-asserted-by":"publisher","first-page":"258","DOI":"10.1109\/IROS47612.2022.9981074","article-title":"Multi-sensor data annotation using sequence-based active learning","author":"Denzler","year":"2022"},{"key":"mlstad1a4fbib18","first-page":"49","article-title":"A practical evaluation of active learning approaches for object detection","volume":"vol 3259","author":"Schneegans","year":"2022"},{"key":"mlstad1a4fbib19","doi-asserted-by":"publisher","first-page":"5966","DOI":"10.1038\/s41467-020-19597-w","article-title":"On-the-fly closed-loop materials discovery via Bayesian active learning","volume":"11","author":"Kusne","year":"2020","journal-title":"Nat. Commun."},{"key":"mlstad1a4fbib20","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1038\/s41524-019-0153-8","article-title":"Active learning in materials science with emphasis on adaptive sampling using uncertainties for targeted design","volume":"5","author":"Lookman","year":"2019","journal-title":"npj Comput. Mater."},{"key":"mlstad1a4fbib21","doi-asserted-by":"publisher","DOI":"10.1002\/adma.202201345","article-title":"Hypothesis learning in automated experiment: application to combinatorial materials libraries","volume":"34","author":"Ziatdinov","year":"2022","journal-title":"Adv. Mater."},{"key":"mlstad1a4fbib22","doi-asserted-by":"publisher","first-page":"327","DOI":"10.1039\/D2DD00113F","article-title":"Quantifying the performance of machine learning models in materials discovery","volume":"2","author":"Borg","year":"2023","journal-title":"Dig. Discovery"},{"key":"mlstad1a4fbib23","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1038\/s41524-023-00967-z","article-title":"Active learning to overcome exponential-wall problem for effective structure prediction of chemical-disordered materials","volume":"9","author":"Yuan","year":"2023","journal-title":"npj Comput. Mater."},{"key":"mlstad1a4fbib24","article-title":"Active learning in open experimental environments: selecting the right information channel(s) based on predictability in deep kernel learning","author":"Ziatdinov","year":"2022"},{"key":"mlstad1a4fbib25","doi-asserted-by":"publisher","DOI":"10.1063\/5.0157644","article-title":"Discovery of structure\u2013property relations for molecules via hypothesis-driven active learning over the chemical space","volume":"1","author":"Ghosh","year":"2023","journal-title":"APL Mach. Learn."},{"key":"mlstad1a4fbib26","doi-asserted-by":"publisher","first-page":"1810","DOI":"10.1021\/acscentsci.3c00502","article-title":"A model ensemble approach enables data-driven property prediction for chemically deconstructable thermosets in the low data regime","volume":"9","author":"AlFaraj","year":"2023","journal-title":"ACS Cent. Sci."},{"key":"mlstad1a4fbib27","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuropharm.2022.109174","article-title":"GAUCHE: a library for Gaussian processes and Bayesian optimisation in chemistry","author":"Griffiths","year":"2022"},{"key":"mlstad1a4fbib28","first-page":"1","article-title":"Distributed active learning with application to battery health management","author":"Chen","year":"2011"},{"key":"mlstad1a4fbib29","doi-asserted-by":"publisher","first-page":"397","DOI":"10.1038\/s41586-020-1994-5","article-title":"Closed-loop optimization of fast-charging protocols for batteries with machine learning","volume":"578","author":"Attia","year":"2020","journal-title":"Nature"},{"key":"mlstad1a4fbib30","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1039\/C3CP52806E","article-title":"Optimal charging profiles for mechanically constrained lithium-ion batteries","volume":"16","author":"Suthar","year":"2014","journal-title":"Phys. Chem. Chem. Phys."},{"key":"mlstad1a4fbib31","doi-asserted-by":"publisher","first-page":"8642","DOI":"10.1109\/CVPR42600.2020.00867","article-title":"Towards visually explaining variational autoencoders","author":"Liu","year":"2020"},{"key":"mlstad1a4fbib32","author":"Liu","year":"2020"},{"key":"mlstad1a4fbib33","first-page":"3565","article-title":"Towards a theoretical understanding of the robustness of variational autoencoders","author":"Camuto","year":"2021"},{"key":"mlstad1a4fbib34","article-title":"Physics and chemistry from parsimonious representations: image analysis via invariant variational autoencoders","author":"Valleti","year":"2023"},{"key":"mlstad1a4fbib35","doi-asserted-by":"publisher","first-page":"e24","DOI":"10.1002\/ail2.24","article-title":"Good practices for Bayesian optimization of high dimensional structured spaces","volume":"2","author":"Siivola","year":"2021","journal-title":"Appl. AI Lett."},{"key":"mlstad1a4fbib36","article-title":"Learning structured output representation using deep conditional generative models","volume":"vol 28","author":"Sohn","year":"2015","journal-title":"Advances in Neural Information Processing Systems"},{"key":"mlstad1a4fbib37","article-title":"Explicitly disentangling image content from translation and rotation with spatial-VAE","volume":"vol 32","author":"Bepler","year":"2019","journal-title":"Advances in Neural Information Processing Systems"},{"key":"mlstad1a4fbib38","doi-asserted-by":"publisher","first-page":"307","DOI":"10.1561\/2200000056","article-title":"An introduction to variational autoencoders","volume":"12","author":"Kingma","year":"2019","journal-title":"Found. Trends\u00ae Mach. Learn."},{"key":"mlstad1a4fbib39","first-page":"97","article-title":"Multidimensional time series anomaly detection: a GRU-based Gaussian mixture variational autoencoder approach","volume":"vol 95","author":"Guo","year":"2018","edition":"ed"},{"key":"mlstad1a4fbib40","doi-asserted-by":"publisher","first-page":"47072","DOI":"10.1109\/ACCESS.2020.2977892","article-title":"Unsupervised anomaly detection of industrial robots using sliding-window convolutional variational autoencoder","volume":"8","author":"Chen","year":"2020","journal-title":"IEEE Access"},{"key":"mlstad1a4fbib41","first-page":"1015","article-title":"Echo-state conditional variational autoencoder for anomaly detection","author":"Suh","year":"2016"},{"key":"mlstad1a4fbib42","article-title":"Variational autoencoder for deep learning of images, labels and captions","volume":"vol 29","author":"Pu","year":"2016","journal-title":"Advances in Neural Information Processing Systems"},{"key":"mlstad1a4fbib43","first-page":"835","article-title":"An uncertain future: forecasting from static images using variational autoencoders","author":"Walker","year":"2016","edition":"ed"},{"key":"mlstad1a4fbib44","first-page":"8655","article-title":"Controlvae: controllable variational autoencoder","author":"Shao","year":"2020"},{"key":"mlstad1a4fbib45","first-page":"1133","article-title":"Deep feature consistent variational autoencoder","author":"Hou","year":"2017"},{"key":"mlstad1a4fbib46","first-page":"442","article-title":"Unsupervised real image super-resolution via generative variational autoencoder","author":"Liu","year":"2020"},{"key":"mlstad1a4fbib47","first-page":"2406","article-title":"Variational image deraining","author":"Du","year":"2020"},{"key":"mlstad1a4fbib48","first-page":"2617","article-title":"Variational autoencoder for low bit-rate image compression","author":"Zhou","year":"2018"},{"key":"mlstad1a4fbib49","doi-asserted-by":"publisher","first-page":"447","DOI":"10.3390\/rs13030447","article-title":"Reduced-complexity end-to-end variational autoencoder for on board satellite image compression","volume":"13","author":"Alves de Oliveira","year":"2021","journal-title":"Remote Sens."},{"key":"mlstad1a4fbib50","doi-asserted-by":"publisher","DOI":"10.1002\/adfm.202100271","article-title":"Latent mechanisms of polarization switching from in situ electron microscopy observations","volume":"32","author":"Ignatans","year":"2022","journal-title":"Adv. Funct. Mater."},{"key":"mlstad1a4fbib51","doi-asserted-by":"publisher","DOI":"10.1063\/5.0105406","article-title":"Unsupervised learning of ferroic variants from atomically resolved STEM images","volume":"12","author":"Valleti","year":"2022","journal-title":"AIP Adv."},{"key":"mlstad1a4fbib52","doi-asserted-by":"publisher","first-page":"6471","DOI":"10.1021\/acsnano.0c08914","article-title":"Disentangling rotational dynamics and ordering transitions in a system of self-organizing protein nanorods via rotationally invariant latent representations","volume":"15","author":"Kalinin","year":"2021","journal-title":"ACS Nano"},{"key":"mlstad1a4fbib53","doi-asserted-by":"publisher","first-page":"eabd5084","DOI":"10.1126\/sciadv.abd5084","article-title":"Exploring order parameters and dynamic processes in disordered systems via variational autoencoders","volume":"7","author":"Kalinin","year":"2021","journal-title":"Sci. Adv."},{"key":"mlstad1a4fbib54","article-title":"Semi-supervised learning of images with strong rotational disorder: assembling nanoparticle libraries","author":"Ziatdinov","year":"2021"},{"key":"mlstad1a4fbib55","doi-asserted-by":"publisher","first-page":"910","DOI":"10.1039\/D2DD00065B","article-title":"Bayesian optimization in continuous spaces via virtual process embeddings","volume":"1","author":"Valleti","year":"2022","journal-title":"Dig. Discovery"},{"key":"mlstad1a4fbib56","first-page":"3576","article-title":"Scalable Gaussian process regression using deep neural networks","author":"Huang","year":"2015"},{"key":"mlstad1a4fbib57","doi-asserted-by":"publisher","first-page":"577","DOI":"10.1039\/C9SC04026A","article-title":"Constrained Bayesian optimization for automatic chemical design using variational autoencoders","volume":"11","author":"Griffiths","year":"2020","journal-title":"Chem. Sci."},{"key":"mlstad1a4fbib58","first-page":"1945","article-title":"Grammar variational autoencoder","author":"Kusner","year":"2017"},{"key":"mlstad1a4fbib59","first-page":"370","article-title":"Deep kernel learning","volume":"vol 51","author":"Wilson","year":"2016","edition":"ed"},{"key":"mlstad1a4fbib60","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1038\/s41524-023-00985-x","article-title":"Learning the right channel in multimodal imaging: automated experiment in piezoresponse force microscopy","volume":"9","author":"Liu","year":"2023","journal-title":"npj Comput. Mater."},{"key":"mlstad1a4fbib61","doi-asserted-by":"publisher","first-page":"13492","DOI":"10.1021\/acsnano.2c05303","article-title":"Bayesian active learning for scanning probe microscopy: from Gaussian processes to hypothesis learning","volume":"16","author":"Ziatdinov","year":"2022","journal-title":"ACS Nano"},{"key":"mlstad1a4fbib62","doi-asserted-by":"publisher","first-page":"9816","DOI":"10.1021\/acs.chemrev.1c00107","article-title":"Combining machine learning and computational chemistry for predictive insights into chemical systems","volume":"121","author":"Keith","year":"2021","journal-title":"Chem. Rev."},{"key":"mlstad1a4fbib63","doi-asserted-by":"publisher","first-page":"p 1","DOI":"10.1609\/aaai.v31i1.11172","article-title":"Deep Gaussian process for crop yield prediction based on remote sensing data","volume":"31","author":"You","year":"2017"},{"key":"mlstad1a4fbib64","doi-asserted-by":"publisher","author":"Garnett","year":"2023","DOI":"10.1017\/9781108348973"},{"key":"mlstad1a4fbib65","doi-asserted-by":"publisher","first-page":"859","DOI":"10.1080\/01621459.2017.1285773","article-title":"Variational inference: a review for statisticians","volume":"112","author":"Blei","year":"2017","journal-title":"J. Am. Stat. Assoc."},{"key":"mlstad1a4fbib66","doi-asserted-by":"publisher","first-page":"4536","DOI":"10.1109\/ACC.2015.7172043","article-title":"Real-time model predictive control for the optimal charging of a lithium-ion battery","author":"Torchio","year":"2015"},{"key":"mlstad1a4fbib67","doi-asserted-by":"publisher","first-page":"477","DOI":"10.1088\/0953-8984\/10\/2\/026","article-title":"Analysis of ferroelectric switching in finite media as a Landau-type phase transition","volume":"10","author":"Dan","year":"1998","journal-title":"J. Phys."},{"key":"mlstad1a4fbib68","doi-asserted-by":"publisher","DOI":"10.1063\/5.0011917","article-title":"Guided search for desired functional responses via Bayesian optimization of generative model: hysteresis loop shape engineering in ferroelectrics","volume":"128","author":"Kalinin","year":"2020","journal-title":"J. Appl. Phys."},{"key":"mlstad1a4fbib69","first-page":"1042","article-title":"Bayesian optimization and attribute adjustment","author":"Eismann","year":"2018"}],"container-title":["Machine Learning: Science and Technology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad1a4f","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad1a4f\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad1a4f","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad1a4f\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad1a4f\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad1a4f\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad1a4f\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad1a4f\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,1,19]],"date-time":"2024-01-19T10:57:23Z","timestamp":1705661843000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad1a4f"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,19]]},"references-count":69,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,1,19]]},"published-print":{"date-parts":[[2024,3,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2632-2153\/ad1a4f","relation":{},"ISSN":["2632-2153"],"issn-type":[{"value":"2632-2153","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,19]]},"assertion":[{"value":"Deep kernel methods learn better: from cards to process optimization","name":"article_title","label":"Article Title"},{"value":"Machine Learning: Science and Technology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2024 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2023-08-15","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2024-01-03","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2024-01-19","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}