{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,27]],"date-time":"2026-08-27T00:05:06Z","timestamp":1787789106644,"version":"build-2784847793"},"reference-count":58,"publisher":"IOP Publishing","issue":"2","license":[{"start":{"date-parts":[[2024,6,12]],"date-time":"2024-06-12T00:00:00Z","timestamp":1718150400000},"content-version":"vor","delay-in-days":11,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,6,12]],"date-time":"2024-06-12T00:00:00Z","timestamp":1718150400000},"content-version":"tdm","delay-in-days":11,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"funder":[{"name":"U.S. Department of Energy, Office of Science, National Quantum Information Science Research Centers, Quantum Systems Accelerator"},{"DOI":"10.13039\/501100022273","name":"Gauss Centre for Supercomputing e.V.","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100022273","id-type":"DOI","asserted-by":"crossref"}]},{"name":"ERC, QuSimCtrl","award":["101113633"],"award-info":[{"award-number":["101113633"]}]},{"name":"Helmholtz Initiative and Networking Fund","award":["VH-NG-1711"],"award-info":[{"award-number":["VH-NG-1711"]}]}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Mach. Learn.: Sci. Technol."],"published-print":{"date-parts":[[2024,6,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>The utility of a quantum computer is highly dependent on the ability to reliably perform accurate quantum logic operations. For finding optimal control solutions, it is of particular interest to explore model-free approaches, since their quality is not constrained by the limited accuracy of theoretical models for the quantum processor\u2014in contrast to many established gate implementation strategies. In this work, we utilize a continuous control reinforcement learning algorithm to design entangling two-qubit gates for superconducting qubits; specifically, our agent constructs cross-resonance and CNOT gates without any prior information about the physical system. Using a simulated environment of fixed-frequency fixed-coupling transmon qubits, we demonstrate the capability to generate novel pulse sequences that outperform the standard cross-resonance gates in both fidelity and gate duration, while maintaining a comparable susceptibility to stochastic unitary noise. We further showcase an augmentation in training and input information that allows our agent to adapt its pulse design abilities to drifting hardware characteristics, importantly, with little to no additional optimization. Our results exhibit clearly the advantages of unbiased adaptive-feedback learning-based optimization methods for transmon gate design.<\/jats:p>","DOI":"10.1088\/2632-2153\/ad4f4d","type":"journal-article","created":{"date-parts":[[2024,5,22]],"date-time":"2024-05-22T22:45:44Z","timestamp":1716417944000},"page":"025066","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["Reinforcement learning pulses for transmon qubit entangling gates"],"prefix":"10.1088","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2189-2090","authenticated-orcid":true,"given":"Ho","family":"Nam Nguyen","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4756-5976","authenticated-orcid":true,"given":"Felix","family":"Motzoi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mekena","family":"Metcalf","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7164-4757","authenticated-orcid":true,"given":"K","family":"Birgitta Whaley","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3688-9599","authenticated-orcid":true,"given":"Marin","family":"Bukov","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2223-8696","authenticated-orcid":false,"given":"Markus","family":"Schmitt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"266","published-online":{"date-parts":[[2024,6,12]]},"reference":[{"key":"mlstad4f4dbib1","author":"Freeman","year":"1998"},{"key":"mlstad4f4dbib2","doi-asserted-by":"publisher","DOI":"10.1103\/RevModPhys.91.045001","article-title":"Shortcuts to adiabaticity: concepts, methods and applications","volume":"91","author":"Gu\u00e9ry-Odelin","year":"2019","journal-title":"Rev. Mod. Phys."},{"key":"mlstad4f4dbib3","doi-asserted-by":"publisher","first-page":"296","DOI":"10.1016\/j.jmr.2004.11.004","article-title":"Optimal control of coupled spin dynamics: design of nmr pulse sequences by gradient ascent algorithms","volume":"172","author":"Khaneja","year":"2005","journal-title":"J. Magn. Reson."},{"key":"mlstad4f4dbib4","doi-asserted-by":"publisher","first-page":"412","DOI":"10.1016\/j.jmr.2011.07.023","article-title":"Second order gradient ascent pulse engineering","volume":"212","author":"de Fouquieres","year":"2011","journal-title":"J. Magn. Reson."},{"key":"mlstad4f4dbib5","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.98.022119","article-title":"Quantum optimal control in a chopped basis: applications in control of Bose-Einstein condensates","volume":"98","author":"S\u00f8rensen","year":"2018","journal-title":"Phys. Rev. A"},{"key":"mlstad4f4dbib6","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.82.042339","article-title":"Reduced phase error through optimized control of a superconducting qubit","volume":"82","author":"Lucero","year":"2010","journal-title":"Phys. Rev. A"},{"key":"mlstad4f4dbib7","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.107.010101","article-title":"Artificial intelligence and machine learning for quantum technologies","volume":"107","author":"Krenn","year":"2023","journal-title":"Phys. Rev. A"},{"key":"mlstad4f4dbib8","doi-asserted-by":"publisher","DOI":"10.1209\/0295-5075\/126\/60002","article-title":"Deep reinforcement learning for quantum gate control","volume":"126","author":"An","year":"2019","journal-title":"Europhys. Lett."},{"key":"mlstad4f4dbib9","doi-asserted-by":"publisher","first-page":"346","DOI":"10.1109\/TAI.2023.3243187","article-title":"Model-free quantum gate design and calibration using deep reinforcement learning","volume":"5","author":"Shindi","year":"2023","journal-title":"IEEE Trans. Artif. Intelligence"},{"key":"mlstad4f4dbib10","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1038\/s41534-019-0241-0","article-title":"Global optimization of quantum dynamics with AlphaZero deep exploration","volume":"6","author":"Dalgaard","year":"2020","journal-title":"npj Quantum Inf."},{"key":"mlstad4f4dbib11","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.103.012404","article-title":"Quantum optimal control of multilevel dissipative quantum systems with reinforcement learning","volume":"103","author":"An","year":"2021","journal-title":"Phys. Rev. A"},{"key":"mlstad4f4dbib12","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevX.12.011059","article-title":"Model-free quantum control with reinforcement learning","volume":"12","author":"Sivak","year":"2022","journal-title":"Phys. Rev. X"},{"key":"mlstad4f4dbib13","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevResearch.5.043002","article-title":"Sample-efficient model-based reinforcement learning for quantum control","volume":"5","author":"Khalid","year":"2023","journal-title":"Phys. Rev. Res."},{"key":"mlstad4f4dbib14","first-page":"pp 2367","article-title":"Deep reinforcement learning for control design of quantum gates","author":"Hu","year":"2022"},{"key":"mlstad4f4dbib15","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1038\/s41534-019-0141-3","article-title":"Universal quantum control through deep reinforcement learning","volume":"5","author":"Niu","year":"2019","journal-title":"npj Quantum Inf."},{"key":"mlstad4f4dbib16","doi-asserted-by":"crossref","DOI":"10.1109\/QCE57702.2023.00146","article-title":"Fast quantum gate design with deep reinforcement learning using real-time feedback on readout signals","author":"Wright","year":"2023"},{"key":"mlstad4f4dbib17","doi-asserted-by":"publisher","DOI":"10.1103\/PRXQuantum.2.040324","article-title":"Experimental deep reinforcement learning for error-robust gate-set design on a superconducting quantum computer","volume":"2","author":"Baum","year":"2021","journal-title":"PRX Quantum"},{"key":"mlstad4f4dbib18","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevB.74.140504","article-title":"Microwave-induced coupling of superconducting qubits","volume":"74","author":"Paraoanu","year":"2006","journal-title":"Phys. Rev. B"},{"key":"mlstad4f4dbib19","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevB.81.134507","article-title":"Fully microwave-tunable universal gates in superconducting qubits with linear couplings and fixed transition frequencies","volume":"81","author":"Rigetti","year":"2010","journal-title":"Phys. Rev. B"},{"key":"mlstad4f4dbib20","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.101.052308","article-title":"Effective hamiltonian models of the cross-resonance gate","volume":"101","author":"Magesan","year":"2020","journal-title":"Phys. Rev. A"},{"key":"mlstad4f4dbib21","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.87.030301","article-title":"Process verification of two-qubit quantum gates by randomized benchmarking","volume":"87","author":"C\u00f3rcoles","year":"2013","journal-title":"Phys. Rev. A"},{"key":"mlstad4f4dbib22","doi-asserted-by":"publisher","first-page":"763","DOI":"10.1038\/nphys1733","article-title":"Selective darkening of degenerate transitions demonstrated with two superconducting quantum bits","volume":"6","author":"de Groot","year":"2010","journal-title":"Nat. Phys."},{"key":"mlstad4f4dbib23","doi-asserted-by":"publisher","DOI":"10.1088\/1367-2630\/14\/7\/073038","article-title":"Selective darkening of degenerate transitions for implementing quantum controlled-not gates","volume":"14","author":"de Groot","year":"2012","journal-title":"New J. Phys."},{"key":"mlstad4f4dbib24","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.93.060302","article-title":"Procedure for systematically tuning up cross-talk in the cross-resonance gate","volume":"93","author":"Sheldon","year":"2016","journal-title":"Phys. Rev. A"},{"key":"mlstad4f4dbib25","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.97.042348","article-title":"Optimized cross-resonance gate for coupled transmon systems","volume":"97","author":"Kirchhoff","year":"2018","journal-title":"Phys. Rev. A"},{"key":"mlstad4f4dbib26","doi-asserted-by":"publisher","DOI":"10.1103\/PRXQuantum.1.020318","article-title":"Reducing unitary and spectator errors in cross resonance with optimized rotary echoes","volume":"1","author":"Sundaresan","year":"2020","journal-title":"PRX Quantum"},{"key":"mlstad4f4dbib27","article-title":"Suppression of coherent errors in cross-resonance gates via recursive drag","author":"Li","year":"2023"},{"key":"mlstad4f4dbib28","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1016\/j.physleta.2007.02.069","article-title":"Fidelity of quantum operations","volume":"367","author":"Pedersen","year":"2007","journal-title":"Phys. Lett. A"},{"key":"mlstad4f4dbib29","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.96.022330","article-title":"Efficient z gates for quantum computing","volume":"96","author":"McKay","year":"2017","journal-title":"Phys. Rev. A"},{"key":"mlstad4f4dbib30","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.86.032324","article-title":"Surface codes: towards practical large-scale quantum computation","volume":"86","author":"Fowler","year":"2012","journal-title":"Phys. Rev. A"},{"key":"mlstad4f4dbib31","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevApplied.21.024018","article-title":"Characterizing non-markovian off-resonant errors in quantum gates","volume":"21","author":"Wei","year":"2024","journal-title":"Phys. Rev. Appl."},{"key":"mlstad4f4dbib32"},{"key":"mlstad4f4dbib33","doi-asserted-by":"publisher","DOI":"10.1088\/0031-8949\/2009\/T137\/014021","article-title":"Optimal generation of fock states in a weakly nonlinear oscillator","volume":"2009","author":"Khani","year":"2009","journal-title":"Phys. Scr."},{"key":"mlstad4f4dbib34","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.97.032306","article-title":"Quantification and characterization of leakage errors","volume":"97","author":"Wood","year":"2018","journal-title":"Phys. Rev. A"},{"key":"mlstad4f4dbib35","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1038\/s41534-017-0036-0","article-title":"Charting the circuit qed design landscape using optimal control theory","volume":"3","author":"Goerz","year":"2017","journal-title":"npj Quantum Inf."},{"key":"mlstad4f4dbib36","article-title":"The two-qutrit states are ordered as (00,01,02,10,\u2026) , the projectors are given by I1=diag(1,1,0,1,1,0,0,0,0) and I2=diag(0,0,1,0,0,1,1,1,1)"},{"key":"mlstad4f4dbib37","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.62.030301","article-title":"Entangling power of quantum evolutions","volume":"62","author":"Zanardi","year":"2000","journal-title":"Phys. Rev. A"},{"key":"mlstad4f4dbib38","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.103.110501","article-title":"Simple pulses for elimination of leakage in weakly nonlinear qubits","volume":"103","author":"Motzoi","year":"2009","journal-title":"Phys. Rev. Lett."},{"key":"mlstad4f4dbib39","doi-asserted-by":"publisher","DOI":"10.1209\/0295-5075\/123\/60001","article-title":"Counteracting systems of diabaticities using drag controls: the status after 10 years (a)","volume":"123","author":"Theis","year":"2018","journal-title":"Europhys. Lett."},{"key":"mlstad4f4dbib40","article-title":"Demonstration of quantum volume 64 on a superconducting quantum computing system","author":"Jurcevic","year":"2020"},{"key":"mlstad4f4dbib41","article-title":"Contextualize me\u2014the case for context in reinforcement learning","author":"Benjamins","year":"2022"},{"key":"mlstad4f4dbib42","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.84.022307","article-title":"Optimal control methods for rapidly time-varying hamiltonians","volume":"84","author":"Motzoi","year":"2011","journal-title":"Phys. Rev. A"},{"key":"mlstad4f4dbib43","article-title":"Continuous control with deep reinforcement learning","author":"Lillicrap","year":"2019"},{"key":"mlstad4f4dbib44","author":"Sutton","year":"2018"},{"key":"mlstad4f4dbib45","article-title":"Addressing function approximation error in actor-critic methods","author":"Fujimoto","year":"2018"},{"key":"mlstad4f4dbib46","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.105.042614","article-title":"Speed limits for two-qubit gates with weakly anharmonic qubits","volume":"105","author":"Ashhab","year":"2022","journal-title":"Phys. Rev. A"},{"key":"mlstad4f4dbib47","doi-asserted-by":"publisher","DOI":"10.1103\/PRXQuantum.3.040311","article-title":"Continuous quantum gate sets and pulse-class meta-optimization","volume":"3","author":"Preti","year":"2022","journal-title":"PRX Quantum"},{"key":"mlstad4f4dbib48","article-title":"Relative changes in system parameters in the following order [\u03a9d0,\u03a9u01,\u03a9d1,\u03a9u10] , [\u03b40,\u03b41] , [\u03b10,\u03b11] , J, are [\u22120.00722279,0.02263578,\u22120.0283148,\u22120.02048592] , [-0.05684035,0.05331411] , [\u22120.04396466,0.05445264] , 0.04959622, where the maximum drift is \u2248\u22125.7%"},{"key":"mlstad4f4dbib49","doi-asserted-by":"publisher","first-page":"2495","DOI":"10.1038\/s41467-022-29940-y","article-title":"Transmon platform for quantum computing challenged by chaotic fluctuations","volume":"13","author":"Berke","year":"2022","journal-title":"Nat. Commun."},{"key":"mlstad4f4dbib50","doi-asserted-by":"publisher","first-page":"7481","DOI":"10.1038\/s41467-022-34851-z","article-title":"High-fidelity qutrit entangling gates for superconducting circuits","volume":"13","author":"Goss","year":"2022","journal-title":"Nat. Commun."},{"key":"mlstad4f4dbib51","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.108.062616","article-title":"Efficient two-qutrit gates in superconducting circuits using parametric coupling","volume":"108","author":"Subramanian","year":"2023","journal-title":"Phys. Rev. A"},{"key":"mlstad4f4dbib52","article-title":"Speed limits of two-qubit gates with qudits","author":"Basyildiz","year":"2023"},{"key":"mlstad4f4dbib53","article-title":"JUWELS: modular tier-0\/1 supercomputer at the J\u00fclich supercomputing centre","volume":"vol 5","author":"J\u00fclich Supercomputing Centre","year":"2019","journal-title":"J. Large-Scale Res. Facilities"},{"key":"mlstad4f4dbib54","article-title":"Rllib: Abstractions for distributed reinforcement learning","author":"Liang","year":"2018"},{"key":"mlstad4f4dbib55","doi-asserted-by":"publisher","first-page":"1234","DOI":"10.1016\/j.cpc.2012.11.019","article-title":"Qutip 2: A python framework for the dynamics of open quantum systems","volume":"184","author":"Johansson","year":"2013","journal-title":"Comput. Phys. Commun."},{"key":"mlstad4f4dbib56","article-title":"Generalized bloch vector and the eigenvalues of a density matrix","author":"Ozols","year":"2007"},{"key":"mlstad4f4dbib57","doi-asserted-by":"publisher","first-page":"7097","DOI":"10.1007\/s00521-019-04191-z","article-title":"Quadratic programming over ellipsoids with applications to constrained linear regression and tensor decomposition","volume":"32","author":"Phan","year":"2020","journal-title":"Neural Comput. Appl."},{"key":"mlstad4f4dbib58","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.93.062320","article-title":"Entangled bloch spheres: bloch matrix and two-qubit state space","volume":"93","author":"Gamel","year":"2016","journal-title":"Phys. Rev. A"}],"container-title":["Machine Learning: Science and Technology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad4f4d","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad4f4d\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad4f4d","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad4f4d\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad4f4d\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad4f4d\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad4f4d\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad4f4d\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,12]],"date-time":"2024-06-12T11:05:30Z","timestamp":1718190330000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/ad4f4d"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,1]]},"references-count":58,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,6,12]]},"published-print":{"date-parts":[[2024,6,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2632-2153\/ad4f4d","relation":{},"ISSN":["2632-2153"],"issn-type":[{"value":"2632-2153","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,1]]},"assertion":[{"value":"Reinforcement learning pulses for transmon qubit entangling gates","name":"article_title","label":"Article Title"},{"value":"Machine Learning: Science and Technology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2024 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2023-12-04","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2024-05-22","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2024-06-12","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}