{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T14:52:48Z","timestamp":1773931968673,"version":"3.50.1"},"reference-count":48,"publisher":"IOP Publishing","issue":"4","license":[{"start":{"date-parts":[[2023,12,21]],"date-time":"2023-12-21T00:00:00Z","timestamp":1703116800000},"content-version":"vor","delay-in-days":20,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,12,21]],"date-time":"2023-12-21T00:00:00Z","timestamp":1703116800000},"content-version":"tdm","delay-in-days":20,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"DOI":"10.13039\/501100009192","name":"Alberta Innovates","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100009192","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"crossref","award":["CREATE-495446-17"],"award-info":[{"award-number":["CREATE-495446-17"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100000196","name":"Canada Foundation for Innovation","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100000196","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001804","name":"Canada Research Chairs","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001804","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Alberta Quantum Major Innovation Fund"},{"DOI":"10.13039\/501100000190","name":"University of Alberta","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100000190","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Mach. Learn.: Sci. Technol."],"published-print":{"date-parts":[[2023,12,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Machine-learning (ML) techniques are emerging as a valuable tool in experimental physics, and among them, reinforcement learning (RL) offers the potential to control high-dimensional, multistage processes in the presence of fluctuating environments. In this experimental work, we apply RL to the preparation of an ultracold quantum gas to realize a consistent and large number of atoms at microkelvin temperatures. This RL agent determines an optimal set of 30 control parameters in a dynamically changing environment that is characterized by 30 sensed parameters. By comparing this method to that of training supervised-learning regression models, as well as to human-driven control schemes, we find that both ML approaches accurately predict the number of cooled atoms and both result in occasional superhuman control schemes. However, only the RL method achieves consistent outcomes, even in the presence of a dynamic environment.<\/jats:p>","DOI":"10.1088\/2632-2153\/ad1437","type":"journal-article","created":{"date-parts":[[2023,12,11]],"date-time":"2023-12-11T17:42:51Z","timestamp":1702316571000},"page":"045057","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["High-dimensional reinforcement learning for optimization and control of ultracold quantum gases"],"prefix":"10.1088","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-5744-4945","authenticated-orcid":true,"given":"N","family":"Milson","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7700-3352","authenticated-orcid":false,"given":"A","family":"Tashchilina","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"T","family":"Ooi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"A","family":"Czarnecka","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Z F","family":"Ahmad","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9156-6100","authenticated-orcid":true,"given":"L J","family":"LeBlanc","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"266","published-online":{"date-parts":[[2023,12,21]]},"reference":[{"key":"mlstad1437bib1","doi-asserted-by":"publisher","first-page":"649","DOI":"10.1038\/s42254-020-0230-4","article-title":"Computer-inspired quantum experiments","volume":"2","author":"Krenn","year":"2020","journal-title":"Nat. Rev. Phys."},{"key":"mlstad1437bib2","doi-asserted-by":"publisher","first-page":"E11231","DOI":"10.1073\/pnas.1716869115","article-title":"Remote optimization of an ultracold atoms experiment by experts and citizen scientists","volume":"115","author":"Heck","year":"2018","journal-title":"Proc. Natl Acad. Sci."},{"key":"mlstad1437bib3","doi-asserted-by":"publisher","first-page":"4360","DOI":"10.1038\/s41467-018-06847-1","article-title":"Multiparameter optimisation of a magneto-optical trap using deep learning","volume":"9","author":"Tranter","year":"2018","journal-title":"Nat. Commun."},{"key":"mlstad1437bib4","doi-asserted-by":"publisher","DOI":"10.1088\/1367-2630\/ac06e6","article-title":"Maximizing the capture velocity of molecular magneto-optical traps with Bayesian optimization","volume":"23","author":"Xu","year":"2021","journal-title":"New J. Phys."},{"key":"mlstad1437bib5","doi-asserted-by":"publisher","DOI":"10.1038\/srep25890","article-title":"Fast machine-learning online optimization of ultra-cold-atom experiments","volume":"6","author":"Wigley","year":"2016","journal-title":"Sci. Rep."},{"key":"mlstad1437bib6","doi-asserted-by":"publisher","first-page":"20435","DOI":"10.1364\/OE.27.020435","article-title":"Non-standard trajectories found by machine learning for evaporative cooling of 87Rb atoms","volume":"27","author":"Nakamura","year":"2019","journal-title":"Opt. Express"},{"key":"mlstad1437bib7","doi-asserted-by":"publisher","DOI":"10.1088\/0256-307X\/37\/10\/103201","article-title":"Active learning approach to optimization of experimental control","volume":"37","author":"Wu","year":"2020","journal-title":"Chin. Phys. Lett."},{"key":"mlstad1437bib8","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.102.011302","article-title":"Machine learning for achieving Bose-Einstein condensation of thulium atoms","volume":"102","author":"Davletov","year":"2020","journal-title":"Phys. Rev. A"},{"key":"mlstad1437bib9","article-title":"Bayesian optimization of Bose-Einstein condensation via evaporative cooling model","author":"Ma","year":"2023"},{"key":"mlstad1437bib10","doi-asserted-by":"publisher","DOI":"10.1088\/2632-2153\/ab6432","article-title":"Applying machine learning optimization methods to the production of a quantum gas","volume":"1","author":"Barker","year":"2020","journal-title":"Mach. Learn.: Sci. Technol."},{"key":"mlstad1437bib11","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevResearch.4.043216","article-title":"Machine-learning-accelerated Bose-Einstein condensation","volume":"4","author":"Vendeiro","year":"2022","journal-title":"Phys. Rev. Res."},{"key":"mlstad1437bib12","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1142\/S0129065704001899","article-title":"Gaussian processes for machine learning","volume":"14","author":"Seeger","year":"2004","journal-title":"Int. J. Neural Syst."},{"key":"mlstad1437bib13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.physrep.2019.03.001","article-title":"A high-bias, low-variance introduction to machine learning for physicists","volume":"810","author":"Mehta","year":"2019","journal-title":"Phys. Rep."},{"key":"mlstad1437bib14","article-title":"Practical Bayesian optimization of machine learning algorithms","volume":"vol 25","author":"Snoek","year":"2012"},{"key":"mlstad1437bib15","doi-asserted-by":"publisher","DOI":"10.1088\/2632-2153\/acd048","article-title":"Closed-loop control of a noisy qubit with reinforcement learning","volume":"4","author":"Ding","year":"2023","journal-title":"Mach. Learn.: Sci. Technol."},{"key":"mlstad1437bib16","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevX.8.031086","article-title":"Reinforcement learning in different phases of quantum control","volume":"8","author":"Bukov","year":"2018","journal-title":"Phys. Rev. X"},{"key":"mlstad1437bib17","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevResearch.3.013034","article-title":"Machine-learning engineering of quantum currents","volume":"3","author":"Haug","year":"2021","journal-title":"Phys. Rev. Res."},{"key":"mlstad1437bib18","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0200455","article-title":"Multiqubit and multilevel quantum reinforcement learning with quantum technologies","volume":"13","author":"C\u00e1rdenas-L\u00f3pez","year":"2018","journal-title":"PLoS One"},{"key":"mlstad1437bib19","doi-asserted-by":"publisher","first-page":"1609","DOI":"10.1038\/s41598-017-01711-6","article-title":"Basic protocols in quantum reinforcement learning with superconducting circuits","volume":"7","author":"Lamata","year":"2017","journal-title":"Sci. Rep."},{"key":"mlstad1437bib20","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6455\/aad62b","article-title":"Machine learning assisted readout of trapped-ion qubits","volume":"51","author":"Seif","year":"2018","journal-title":"J. Phys. B: At. Mol. Opt. Phys."},{"key":"mlstad1437bib21","doi-asserted-by":"publisher","DOI":"10.1088\/2632-2153\/abf0f6","article-title":"Playing optical tweezers with deep reinforcement learning: in virtual, physical and augmented environments","volume":"2","author":"Praeger","year":"2021","journal-title":"Mach. Learn.: Sci. Technol."},{"key":"mlstad1437bib22","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.118.158004","article-title":"Flow navigation by smart microswimmers via reinforcement learning","volume":"118","author":"Colabrese","year":"2017","journal-title":"Phys. Rev. Lett."},{"key":"mlstad1437bib23","first-page":"67","article-title":"Making, probing and understanding Bose-Einstein condensates","author":"Ketterle","year":"1999"},{"key":"mlstad1437bib24","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.79.063631","article-title":"Rapid production of 87Rb Bose-Einstein condensates in a combined magnetic and optical potential","volume":"79","author":"Lin","year":"2009","journal-title":"Phys. Rev. A"},{"key":"mlstad1437bib25","doi-asserted-by":"publisher","DOI":"10.1088\/1367-2630\/abf1d9","article-title":"Storing short single-photon-level optical pulses in Bose\u2013Einstein condensates for high-performance quantum memory","volume":"23","author":"Saglamyurek","year":"2021","journal-title":"New J. Phys."},{"key":"mlstad1437bib26","doi-asserted-by":"publisher","first-page":"338","DOI":"10.1109\/TAC.2005.844079","article-title":"Bandit problems with side observations","volume":"50","author":"Wang","year":"2005","journal-title":"IEEE Trans. Autom. Control"},{"key":"mlstad1437bib27","author":"Sutton","year":"2018"},{"key":"mlstad1437bib28","first-page":"489","article-title":"Heteroscedastic Gaussian process regression","author":"Le","year":"2005"},{"key":"mlstad1437bib29","first-page":"249","article-title":"Understanding the difficulty of training deep feedforward neural networks","author":"Glorot","year":"2010"},{"key":"mlstad1437bib30","article-title":"Adam: a method for stochastic optimization","author":"Kingma","year":"2014"},{"key":"mlstad1437bib31","article-title":"Self-normalizing neural networks","volume":"vol 30","author":"Klambauer","year":"2017"},{"key":"mlstad1437bib32","doi-asserted-by":"publisher","first-page":"877","DOI":"10.1007\/s00041-008-9045-x","article-title":"Enhancing sparsity by reweighted L 1 minimization","volume":"14","author":"Candes","year":"2008","journal-title":"J. Fourier Anal. Appl."},{"key":"mlstad1437bib33","article-title":"Policy gradient methods for reinforcement learning with function approximation","volume":"vol 12","author":"Sutton","year":"1999"},{"key":"mlstad1437bib34","doi-asserted-by":"publisher","first-page":"eabb9764","DOI":"10.1126\/scirobotics.abb9764","article-title":"An adaptive deep reinforcement learning framework enables curling robots with human-like performance in real-world conditions","volume":"5","author":"Won","year":"2020","journal-title":"Sci. Robot."},{"key":"mlstad1437bib35","doi-asserted-by":"publisher","first-page":"293","DOI":"10.1007\/BF00992699","article-title":"Self-improving reactive agents based on reinforcement learning, planning and teaching","volume":"8","author":"Lin","year":"1992","journal-title":"Mach. Learn."},{"key":"mlstad1437bib36","article-title":"Gaussian error linear units (GELUs)","author":"Hendrycks","year":"2016"},{"key":"mlstad1437bib37","first-page":"492","article-title":"Robust estimation of a location parameter","author":"Huber","year":"1992"},{"key":"mlstad1437bib38","doi-asserted-by":"publisher","first-page":"1190","DOI":"10.1137\/0916069","article-title":"A limited memory algorithm for bound constrained optimization","volume":"16","author":"Byrd","year":"1995","journal-title":"SIAM J. Sci. Comput."},{"key":"mlstad1437bib39","doi-asserted-by":"publisher","first-page":"308","DOI":"10.1093\/comjnl\/7.4.308","article-title":"A simplex method for function minimization","volume":"7","author":"Nelder","year":"1965","journal-title":"Comput. J."},{"key":"mlstad1437bib40","first-page":"1","article-title":"All models are wrong, but many are useful: learning a variable\u2019s importance by studying an entire class of prediction models simultaneously","volume":"20","author":"Fisher","year":"2019","journal-title":"J. Mach. Learn. Res."},{"key":"mlstad1437bib41","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"mlstad1437bib42","first-page":"2829","article-title":"Continuous deep Q-learning with model-based acceleration","author":"Gu","year":"2016"},{"key":"mlstad1437bib43","article-title":"Reinforcement learning in continuous action spaces through sequential monte carlo methods","volume":"vol 20","author":"Lazaric","year":"2007"},{"key":"mlstad1437bib44","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.62.043403","article-title":"Sensitive magnetometry based on nonlinear magneto-optical rotation","volume":"62","author":"Budker","year":"2000","journal-title":"Phys. Rev. A"},{"key":"mlstad1437bib45","doi-asserted-by":"publisher","DOI":"10.1088\/2632-2153\/abffe7","article-title":"Unsupervised machine learning of topological phase transitions from experimental data","volume":"2","author":"K\u00e4ming","year":"2021","journal-title":"Mach. Learn.: Sci. Technol."},{"key":"mlstad1437bib46","doi-asserted-by":"publisher","first-page":"37786","DOI":"10.1364\/OE.473770","article-title":"Observing a topological phase transition with deep neural networks from experimental images of ultracold atoms","volume":"30","author":"Zhao","year":"2022","journal-title":"Opt. Express"},{"key":"mlstad1437bib47","doi-asserted-by":"publisher","DOI":"10.1116\/5.0126745","article-title":"Bi-color atomic beam slower and magnetic field compensation for ultracold gases","volume":"4","author":"Li","year":"2022","journal-title":"AVS Quantum Sci."},{"key":"mlstad1437bib48","article-title":"Reinforcement learning in ultracold atom experiments","author":"Reinschmidt","year":"2023"}],"container-title":["Machine Learning: Science and Technology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad1437","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad1437\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad1437","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad1437\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad1437\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad1437\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad1437\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad1437\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,21]],"date-time":"2023-12-21T03:17:55Z","timestamp":1703128675000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad1437"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,1]]},"references-count":48,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2023,12,21]]},"published-print":{"date-parts":[[2023,12,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2632-2153\/ad1437","relation":{"has-review":[{"id-type":"doi","id":"10.1088\/2632-2153\/AD1437\/v1\/decision1","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2632-2153\/AD1437\/v2\/decision1","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2632-2153\/AD1437\/v2\/response1","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2632-2153\/AD1437\/v2\/review1","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2632-2153\/AD1437\/v2\/review2","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2632-2153\/AD1437\/v1\/review1","asserted-by":"object"},{"id-type":"doi","id":"10.1088\/2632-2153\/AD1437\/v1\/review2","asserted-by":"object"}]},"ISSN":["2632-2153"],"issn-type":[{"value":"2632-2153","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,1]]},"assertion":[{"value":"High-dimensional reinforcement learning for optimization and control of ultracold quantum gases","name":"article_title","label":"Article Title"},{"value":"Machine Learning: Science and Technology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2023 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2023-09-14","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2023-12-11","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2023-12-21","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}