{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T05:57:09Z","timestamp":1780466229224,"version":"3.54.1"},"reference-count":74,"publisher":"IOP Publishing","issue":"3","license":[{"start":{"date-parts":[[2021,4,22]],"date-time":"2021-04-22T00:00:00Z","timestamp":1619049600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,4,22]],"date-time":"2021-04-22T00:00:00Z","timestamp":1619049600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Mach. Learn.: Sci. Technol."],"published-print":{"date-parts":[[2021,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Control of the stochastic dynamics of a quantum system is indispensable in fields such as quantum information processing and metrology. However, there is no general ready-made approach to the design of efficient control strategies. Here, we propose a framework for the automated design of control schemes based on differentiable programming. We apply this approach to the state preparation and stabilization of a qubit subjected to homodyne detection. To this end, we formulate the control task as an optimization problem where the loss function quantifies the distance from the target state, and we employ neural networks (NNs) as controllers. The system\u2019s time evolution is governed by a stochastic differential equation (SDE). To implement efficient training, we backpropagate the gradient information from the loss function through the SDE solver using adjoint sensitivity methods. As a first example, we feed the quantum state to the controller and focus on different methods of obtaining gradients. As a second example, we directly feed the homodyne detection signal to the controller. The instantaneous value of the homodyne current contains only very limited information on the actual state of the system, masked by unavoidable photon-number fluctuations. Despite the resulting poor signal-to-noise ratio, we can train our controller to prepare and stabilize the qubit to a target state with a mean fidelity of around 85%. We also compare the solutions found by the NN to a hand-crafted control strategy.<\/jats:p>","DOI":"10.1088\/2632-2153\/abec22","type":"journal-article","created":{"date-parts":[[2021,3,4]],"date-time":"2021-03-04T22:31:44Z","timestamp":1614897104000},"page":"035004","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Control of stochastic quantum dynamics by differentiable programming"],"prefix":"10.1088","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2684-4984","authenticated-orcid":false,"given":"Frank","family":"Sch\u00e4fer","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8455-020X","authenticated-orcid":false,"given":"Pavel","family":"Sekatski","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0162-3261","authenticated-orcid":false,"given":"Martin","family":"Koppenh\u00f6fer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christoph","family":"Bruder","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4575-7723","authenticated-orcid":false,"given":"Michal","family":"Kloc","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"266","published-online":{"date-parts":[[2021,4,22]]},"reference":[{"key":"mlstabec22bib1","author":"D\u2019Alessandro","year":"2008"},{"key":"mlstabec22bib2","author":"Wiseman","year":"2009"},{"key":"mlstabec22bib3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1140\/epjd\/e2015-60464-1","article-title":"Training Schr\u00f6dinger\u2019s cat: quantum optimal control","volume":"69","author":"Glaser","year":"2015","journal-title":"Eur. Phys. J. D"},{"key":"mlstabec22bib4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.physrep.2017.02.003","article-title":"Quantum feedback: theory, experiments and applications","volume":"679","author":"Zhang","year":"2017","journal-title":"Phys. Rep."},{"key":"mlstabec22bib5","author":"Sutton","year":"2018","edition":"2nd edn"},{"key":"mlstabec22bib6","article-title":"Continuous control with deep reinforcement learning","author":"Lillicrap","year":"2016"},{"key":"mlstabec22bib7","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1038\/s41534-019-0141-3","article-title":"Universal quantum control through deep reinforcement learning","volume":"5","author":"Niu","year":"2019","journal-title":"npj Quantum Inf."},{"key":"mlstabec22bib8","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevB.98.224305","article-title":"Reinforcement learning for autonomous preparation of Floquet-engineered states: inverting the quantum Kapitza oscillator","volume":"98","author":"Bukov","year":"2018","journal-title":"Phys. Rev. B"},{"key":"mlstabec22bib9","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevX.8.031086","article-title":"Reinforcement learning in different phases of quantum control","volume":"8","author":"Bukov","year":"2018","journal-title":"Phys. Rev. X"},{"key":"mlstabec22bib10","doi-asserted-by":"publisher","DOI":"10.1088\/2632-2153\/ab9802","article-title":"A differentiable programming method for quantum control","volume":"1","author":"Sch\u00e4fer","year":"2020","journal-title":"Machine Learn.: Sci. Technol."},{"key":"mlstabec22bib11","doi-asserted-by":"crossref","DOI":"10.21203\/rs.3.rs-55125\/v1","article-title":"Universal differential equations for scientific machine learning","author":"Rackauckas","year":"2020"},{"key":"mlstabec22bib12","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.95.042318","article-title":"Speedup for quantum optimal control from automatic differentiation based on graphics processing units","volume":"95","author":"Leung","year":"2017","journal-title":"Phys. Rev. A"},{"key":"mlstabec22bib13","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.99.052327","article-title":"Gradient-based optimal control of open quantum systems using quantum trajectories and automatic differentiation","volume":"99","author":"Abdelhafez","year":"2019","journal-title":"Phys. Rev. A"},{"key":"mlstabec22bib14","article-title":"Generalized physics-informed learning through language-wide differentiable programming","author":"Rackauckas","year":"2020"},{"key":"mlstabec22bib15","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevX.9.031041","article-title":"Differentiable programming tensor networks","volume":"9","author":"Liao","year":"2019","journal-title":"Phys. Rev. X"},{"key":"mlstabec22bib16","article-title":"Diffeqflux.jl\u2014a Julia library for neural differential equations","author":"Rackauckas","year":"2019"},{"key":"mlstabec22bib17","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.99.042327","article-title":"Learning robust and high-precision quantum controls","volume":"99","author":"Wu","year":"2019","journal-title":"Phys. Rev. A"},{"key":"mlstabec22bib18","author":"Coopmans","year":"2020"},{"key":"mlstabec22bib19","author":"Breuer","year":"2002"},{"key":"mlstabec22bib20","doi-asserted-by":"publisher","first-page":"131","DOI":"10.1140\/epjd\/e2002-00217-9","article-title":"Optical phase-space reconstruction of mirror motion at the attometer level","volume":"22","author":"Briant","year":"2003","journal-title":"Eur. Phys. J. D"},{"key":"mlstabec22bib21","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.111.163602","article-title":"Quantum-limited mirror-motion estimation","volume":"111","author":"Iwasawa","year":"2013","journal-title":"Phys. Rev. Lett."},{"key":"mlstabec22bib22","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.114.223601","article-title":"Optimal state estimation for cavity optomechanical systems","volume":"114","author":"Wieczorek","year":"2015","journal-title":"Phys. Rev. Lett."},{"key":"mlstabec22bib23","doi-asserted-by":"publisher","first-page":"548","DOI":"10.1103\/PhysRevLett.70.548","article-title":"Quantum theory of optical feedback via homodyne detection","volume":"70","author":"Wiseman","year":"1993","journal-title":"Phys. Rev. Lett."},{"key":"mlstabec22bib24","doi-asserted-by":"publisher","first-page":"688","DOI":"10.1103\/PhysRevLett.80.688","article-title":"Optomechanical cooling of a macroscopic oscillator by homodyne feedback","volume":"80","author":"Mancini","year":"1998","journal-title":"Phys. Rev. Lett."},{"key":"mlstabec22bib25","doi-asserted-by":"publisher","first-page":"4877","DOI":"10.1103\/PhysRevA.57.4877","article-title":"Quantum control of atomic systems by homodyne detection and feedback","volume":"57","author":"Hofmann","year":"1998","journal-title":"Phys. Rev. A"},{"key":"mlstabec22bib26","doi-asserted-by":"publisher","first-page":"2700","DOI":"10.1103\/PhysRevA.60.2700","article-title":"Feedback control of quantum systems using continuous state estimation","volume":"60","author":"Doherty","year":"1999","journal-title":"Phys. Rev. A"},{"key":"mlstabec22bib27","doi-asserted-by":"publisher","first-page":"325","DOI":"10.1038\/nature14672","article-title":"Measurement-based control of a mechanical oscillator at its thermal decoherence rate","volume":"524","author":"Wilson","year":"2015","journal-title":"Nature"},{"key":"mlstabec22bib28","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.93.120408","article-title":"Entanglement within the quantum trajectory description of open quantum systems","volume":"93","author":"Nha","year":"2004","journal-title":"Phys. Rev. Lett."},{"key":"mlstabec22bib29","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.105.210502","article-title":"Entanglement dynamics in open two-qubit systems via diffusive quantum trajectories","volume":"105","author":"Viviescas","year":"2010","journal-title":"Phys. Rev. Lett."},{"key":"mlstabec22bib30","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.97.063812","article-title":"Unraveling nonclassicality in the optomechanical instability","volume":"97","author":"Koppenh\u00f6fer","year":"2018","journal-title":"Phys. Rev. A"},{"key":"mlstabec22bib31","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevResearch.2.013071","article-title":"Heralded dissipative preparation of nonclassical states in a Kerr oscillator","volume":"2","author":"Koppenh\u00f6fer","year":"2020","journal-title":"Phys. Rev. Res."},{"key":"mlstabec22bib32","doi-asserted-by":"publisher","first-page":"5158","DOI":"10.1103\/PhysRevLett.83.5158","article-title":"Proposal for teleportation of an atomic state via cavity decay","volume":"83","author":"Bose","year":"1999","journal-title":"Phys. Rev. Lett."},{"key":"mlstabec22bib33","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.94.042334","article-title":"Quantum teleportation with continuous measurements","volume":"94","author":"Greplova","year":"2016","journal-title":"Phys. Rev. A"},{"key":"mlstabec22bib34","doi-asserted-by":"publisher","first-page":"1926","DOI":"10.1038\/s41467-018-04372-9","article-title":"Dynamics of a qubit while simultaneously monitoring its relaxation and dephasing","volume":"9","author":"Ficheux","year":"2018","journal-title":"Nat. Commun."},{"key":"mlstabec22bib35","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1038\/nature11505","article-title":"Stabilizing Rabi oscillations in a superconducting qubit using quantum feedback","volume":"490","author":"Vijay","year":"2012","journal-title":"Nature"},{"key":"mlstabec22bib36","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.89.133602","article-title":"Adaptive homodyne measurement of optical phase","volume":"89","author":"Armen","year":"2002","journal-title":"Phys. Rev. Lett."},{"key":"mlstabec22bib37","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/ncomms11527","article-title":"Mapping quantum state dynamics in spontaneous emission","volume":"7","author":"Naghiloo","year":"2016","journal-title":"Nat. Commun."},{"key":"mlstabec22bib38","doi-asserted-by":"publisher","first-page":"2199","DOI":"10.1137\/060651239","article-title":"An introduction to quantum filtering","volume":"46","author":"Bouten","year":"2007","journal-title":"SIAM J. Control Optim."},{"key":"mlstabec22bib39","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevX.10.011006","article-title":"Using a recurrent neural network to reconstruct quantum dynamics of a superconducting qubit from physical observations","volume":"10","author":"Flurin","year":"2020","journal-title":"Phys. Rev. X"},{"key":"mlstabec22bib40","doi-asserted-by":"publisher","first-page":"15","DOI":"10.5334\/jors.151","article-title":"Differentialequations.jl\u2014a performant and feature-rich ecosystem for solving differential equations in Julia","volume":"5","author":"Rackauckas","year":"2017","journal-title":"J. Open Res. Softw."},{"key":"mlstabec22bib41","doi-asserted-by":"publisher","first-page":"2731","DOI":"10.3934\/dcdsb.2017133","article-title":"Adaptive methods for stochastic differential equations via natural embeddings and rejection sampling with memory","volume":"22","author":"Rackauckas","year":"2017","journal-title":"Discrete Contin. Dyn. Syst. B"},{"key":"mlstabec22bib42","first-page":"pp 1","article-title":"Stability-optimized high order methods and stiffness detection for pathwise stiff stochastic differential equations","author":"Rackauckas","year":"2020"},{"key":"mlstabec22bib43","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.84.022326","article-title":"Chopped random-basis quantum optimization","volume":"84","author":"Caneva","year":"2011","journal-title":"Phys. Rev. A"},{"key":"mlstabec22bib44","doi-asserted-by":"publisher","first-page":"75","DOI":"10.1145\/84537.84552","article-title":"Likelihood ratio gradient estimation for stochastic systems","volume":"33","author":"Glynn","year":"1990","journal-title":"Commun. ACM"},{"key":"mlstabec22bib45","doi-asserted-by":"publisher","first-page":"1216","DOI":"10.1137\/0329064","article-title":"A Monte Carlo method for sensitivity analysis and parametric optimization of nonlinear stochastic systems","volume":"29","author":"Yang","year":"1991","journal-title":"SIAM J. Control Optim."},{"key":"mlstabec22bib46","doi-asserted-by":"publisher","first-page":"413","DOI":"10.1016\/0377-2217(95)00107-7","article-title":"Optimization and sensitivity analysis of computer simulation models by the score function method","volume":"88","author":"Kleijnen","year":"1996","journal-title":"Eur. J. Oper. Res."},{"key":"mlstabec22bib47","doi-asserted-by":"publisher","first-page":"229","DOI":"10.1007\/BF00992696","article-title":"Simple statistical gradient-following algorithms for connectionist reinforcement learning","volume":"8","author":"Williams","year":"1992","journal-title":"Mach. Learn."},{"key":"mlstabec22bib48","first-page":"5595","article-title":"Automatic differentiation in machine learning: a survey","volume":"18","author":"Baydin","year":"2017","journal-title":"J. Machine Learn. Res."},{"key":"mlstabec22bib49","doi-asserted-by":"publisher","first-page":"463","DOI":"10.1145\/355586.364791","article-title":"A simple automatic derivative evaluation program","volume":"7","author":"Wengert","year":"1964","journal-title":"Commun. ACM"},{"key":"mlstabec22bib50","article-title":"A comparison of automatic differentiation and continuous sensitivity analysis for derivatives of differential equation solutions","author":"Rackauckas","year":"2018"},{"key":"mlstabec22bib51","author":"Pontryagin","year":"2018"},{"key":"mlstabec22bib52","article-title":"Scalable gradients for stochastic differential equations","author":"Li","year":"2020"},{"key":"mlstabec22bib53","article-title":"Neural SDEs made easy: SDEs are infinite-dimensional GANs","author":"Kidger","year":"2021"},{"key":"mlstabec22bib54","author":"Kloeden","year":"2013"},{"key":"mlstabec22bib55","article-title":"High weak order solvers and adjoint sensitivity analysis for stochastic differential equations","author":"Sch\u00e4fer","year":"2020"},{"key":"mlstabec22bib56","article-title":"Julia: a fast dynamic language for technical computing","author":"Bezanson","year":"2012"},{"key":"mlstabec22bib57","article-title":"Continuous-discrete smoothing of diffusions","author":"van der Meulen","year":"2017"},{"key":"mlstabec22bib58","article-title":"Automatic backward filtering forward guiding for Markov processes and graphical models","author":"van der Meulen","year":"2020"},{"key":"mlstabec22bib59","doi-asserted-by":"publisher","first-page":"S237","DOI":"10.1088\/1464-4266\/7\/10\/006","article-title":"Hamilton\u2013Jacobi\u2013Bellman equations for quantum optimal feedback control","volume":"7","author":"Gough","year":"2005","journal-title":"J. Opt. B: Quantum Semiclass. Opt."},{"key":"mlstabec22bib60","doi-asserted-by":"publisher","first-page":"1339","DOI":"10.1016\/j.jcp.2018.08.029","article-title":"DGM: a deep learning algorithm for solving partial differential equations","volume":"375","author":"Sirignano","year":"2018","journal-title":"J. Comput. Phys."},{"key":"mlstabec22bib61","article-title":"The Koopman expectation: an operator theoretic method for efficient analysis and optimization of uncertain hybrid dynamical systems","author":"Gerlach","year":"2020"},{"key":"mlstabec22bib62","doi-asserted-by":"publisher","first-page":"1726","DOI":"10.1109\/TAC.2018.2872274","article-title":"Analysis and control of stochastic systems using semidefinite programming over moments","volume":"64","author":"Lamperski","year":"2018","journal-title":"IEEE Trans. Autom. Control"},{"key":"mlstabec22bib63","article-title":"Quantum parameter estimation with a neural network","author":"Greplova","year":"2017"},{"key":"mlstabec22bib64","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevResearch.1.033092","article-title":"Hamiltonian learning for quantum error correction","volume":"1","author":"Valenti","year":"2019","journal-title":"Phys. Rev. Res."},{"key":"mlstabec22bib65","doi-asserted-by":"publisher","DOI":"10.1088\/2058-9565\/ab18d5","article-title":"Stochastic estimation of dynamical variables","volume":"4","author":"Krastanov","year":"2019","journal-title":"Quantum Sci. Technol."},{"key":"mlstabec22bib66","author":"Sch\u00e4fer","year":"2021"},{"key":"mlstabec22bib67","doi-asserted-by":"publisher","first-page":"3932","DOI":"10.1073\/pnas.1517384113","article-title":"Discovering governing equations from data by sparse identification of nonlinear dynamical systems","volume":"113","author":"Brunton","year":"2016","journal-title":"Proc. Natl Acad. Sci."},{"key":"mlstabec22bib68","doi-asserted-by":"publisher","DOI":"10.1063\/1.5018409","article-title":"Sparse learning of stochastic dynamical equations","volume":"148","author":"Boninsegna","year":"2018","journal-title":"J. Chem. Phys."},{"key":"mlstabec22bib69","doi-asserted-by":"publisher","DOI":"10.1098\/rspa.2018.0335","article-title":"Sparse identification of nonlinear dynamics for model predictive control in the low-data limit","volume":"474","author":"Kaiser","year":"2018","journal-title":"Proc. R. Soc. A"},{"key":"mlstabec22bib70","article-title":"Discovering symbolic models from deep learning with inductive biases","author":"Cranmer","year":"2020"},{"key":"mlstabec22bib71","article-title":"Zygote: a differentiable programming system to bridge machine learning and scientific computing","author":"Innes","year":"2019"},{"key":"mlstabec22bib72","first-page":"6571","author":"Chen","year":"2018"},{"key":"mlstabec22bib73","author":"Johnson","year":"2007"},{"key":"mlstabec22bib74","first-page":"9847","author":"Jia","year":"2019"}],"container-title":["Machine Learning: Science and Technology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abec22","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abec22\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abec22","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abec22\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abec22\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abec22\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abec22\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abec22\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,2,5]],"date-time":"2022-02-05T00:40:16Z","timestamp":1644021616000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abec22"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,22]]},"references-count":74,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2021,4,22]]},"published-print":{"date-parts":[[2021,9,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2632-2153\/abec22","relation":{},"ISSN":["2632-2153"],"issn-type":[{"value":"2632-2153","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,22]]},"assertion":[{"value":"Control of stochastic quantum dynamics by differentiable programming","name":"article_title","label":"Article Title"},{"value":"Machine Learning: Science and Technology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2021 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2020-12-30","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2021-03-04","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2021-04-22","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}