{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,30]],"date-time":"2026-10-30T03:47:16Z","timestamp":1793332036202,"version":"4.5.0"},"reference-count":69,"publisher":"IOP Publishing","issue":"2","license":[{"start":{"date-parts":[[2020,12,28]],"date-time":"2020-12-28T00:00:00Z","timestamp":1609113600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,12,28]],"date-time":"2020-12-28T00:00:00Z","timestamp":1609113600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/iopscience.iop.org\/info\/page\/text-and-data-mining"}],"content-domain":{"domain":["iopscience.iop.org"],"crossmark-restriction":false},"short-container-title":["Mach. Learn.: Sci. Technol."],"published-print":{"date-parts":[[2021,6,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Topological error correcting codes, and particularly the surface code, currently provide the most feasible road-map towards large-scale fault-tolerant quantum computation. As such, obtaining fast and flexible decoding algorithms for these codes, within the experimentally realistic and challenging context of faulty syndrome measurements, without requiring any final read-out of the physical qubits, is of critical importance. In this work, we show that the problem of decoding such codes can be naturally reformulated as a process of repeated interactions between a decoding agent and a code environment, to which the machinery of reinforcement learning can be applied to obtain decoding agents. While in principle this framework can be instantiated with environments modelling circuit level noise, we take a first step towards this goal by using deepQ learning to obtain decoding agents for a variety of simplified phenomenological noise models, which yield faulty syndrome measurements without including the propagation of errors which arise in full circuit level noise models.<\/jats:p>","DOI":"10.1088\/2632-2153\/abc609","type":"journal-article","created":{"date-parts":[[2020,10,29]],"date-time":"2020-10-29T22:31:26Z","timestamp":1604010686000},"page":"025005","update-policy":"https:\/\/doi.org\/10.1088\/crossmark-policy","source":"Crossref","is-referenced-by-count":70,"title":["Reinforcement learning decoders for fault-tolerant quantum computation"],"prefix":"10.1088","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6202-8864","authenticated-orcid":false,"given":"Ryan","family":"Sweke","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Markus S","family":"Kesselring","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0323-0031","authenticated-orcid":false,"given":"Evert P L","family":"van Nieuwenburg","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jens","family":"Eisert","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"266","published-online":{"date-parts":[[2020,12,28]]},"reference":[{"key":"mlstabc609bib1","doi-asserted-by":"publisher","first-page":"307","DOI":"10.1103\/RevModPhys.87.307","article-title":"Quantum error correcton for quantum memories","volume":"87","author":"Terhal","year":"2015","journal-title":"Rev. Mod. Phys."},{"key":"mlstabc609bib2","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1038\/nature23460","article-title":"Roads towards fault-tolerant universal quantum computation","volume":"549","author":"Campbell","year":"2017","journal-title":"Nature"},{"key":"mlstabc609bib3","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1016\/S0003-4916(02)00018-0","article-title":"Fault-tolerant quantum computation by anyons","volume":"303","author":"Kitaev","year":"2003","journal-title":"Ann. Phys."},{"key":"mlstabc609bib4","doi-asserted-by":"publisher","first-page":"4452","DOI":"10.1063\/1.1499754","article-title":"Topological quantum memory","volume":"43","author":"Dennis","year":"2002","journal-title":"J. Math. Phys."},{"key":"mlstabc609bib5","author":"Preskill","year":"2017"},{"key":"mlstabc609bib6","doi-asserted-by":"publisher","first-page":"1083","DOI":"10.1103\/RevModPhys.80.1083","article-title":"Non-Abelian anyons and topological quantum computation","volume":"80","author":"Nayak","year":"2008","journal-title":"Rev. Mod. Phys."},{"key":"mlstabc609bib7","author":"Pachos","year":"2012"},{"key":"mlstabc609bib8","doi-asserted-by":"publisher","DOI":"10.1103\/RevModPhys.88.045005","article-title":"Quantum memories at finite temperature","volume":"88","author":"Brown","year":"2016","journal-title":"Rev. Mod. Phys."},{"key":"mlstabc609bib9","doi-asserted-by":"publisher","first-page":"382","DOI":"10.1038\/nature10786","article-title":"Realization of three-qubit quantum error correction with superconducting circuits","volume":"482","author":"Reed","year":"2012","journal-title":"Nature"},{"key":"mlstabc609bib10","doi-asserted-by":"publisher","first-page":"500","DOI":"10.1038\/nature13171","article-title":"Superconducting quantum circuits at the surface code threshold for fault tolerance","volume":"508","author":"Barends","year":"2014","journal-title":"Nature"},{"key":"mlstabc609bib11","doi-asserted-by":"publisher","first-page":"302","DOI":"10.1126\/science.1253742","article-title":"Quantum computations on a topologically encoded qubit","volume":"345","author":"Nigg","year":"2014","journal-title":"Science"},{"key":"mlstabc609bib12","doi-asserted-by":"publisher","first-page":"6979","DOI":"10.1038\/ncomms7979","article-title":"Demonstration of a quantum error detection code using a square lattice of four superconducting qubits","volume":"6","author":"C\u00f3rcoles","year":"2015","journal-title":"Nat. Comm."},{"key":"mlstabc609bib13","doi-asserted-by":"publisher","first-page":"206","DOI":"10.1038\/nature17162","article-title":"Exponential protection of zero modes in Majorana islands","volume":"531","author":"Albrecht","year":"2016","journal-title":"Nature"},{"key":"mlstabc609bib14","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.117.210505","article-title":"Demonstration of weight-four parity measurements in the surface code architecture","volume":"117","author":"Takita","year":"2016","journal-title":"Phys. Rev. Lett."},{"key":"mlstabc609bib15","doi-asserted-by":"publisher","DOI":"10.1126\/sciadv.1701074","article-title":"Fault-tolerant quantum error detection","volume":"3","author":"Linke","year":"2017","journal-title":"Sci. Adv."},{"key":"mlstabc609bib16","article-title":"Low overhead quantum computation using lattice surgery","author":"Fowler","year":"2018"},{"key":"mlstabc609bib17","article-title":"A game of surface codes: Large-scale quantum computing with lattice surgery","author":"Litinski","year":"2018"},{"key":"mlstabc609bib18","doi-asserted-by":"publisher","DOI":"10.1088\/1367-2630\/aad1ea","article-title":"The European quantum technologies roadmap","volume":"20","author":"Acin","year":"2018","journal-title":"New J. Phys."},{"key":"mlstabc609bib19","article-title":"Minimum weight perfect matching of fault-tolerant topological quantum error correction in average o(1) parallel time","author":"Fowler","year":"2013"},{"key":"mlstabc609bib20","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.104.050504","article-title":"Fast decoders for topological quantum codes","volume":"104","author":"Duclos-Cianci","year":"2010","journal-title":"Phys. Rev. Lett."},{"key":"mlstabc609bib21","doi-asserted-by":"publisher","DOI":"10.1038\/npjqi.2015.10","article-title":"Cellular-automaton decoders for topological quantum memories","volume":"1","author":"Herold","year":"2015","journal-title":"Npj Quant. Inf."},{"key":"mlstabc609bib22","doi-asserted-by":"publisher","DOI":"10.1088\/1367-2630\/aa7099","article-title":"Cellular automaton decoders of topological quantum memories in the fault tolerant setting","volume":"19","author":"Herold","year":"2017","journal-title":"New J. Phys."},{"key":"mlstabc609bib23","article-title":"Cellular-automaton decoders with provable thresholds for topological codes","author":"Kubica","year":"2018"},{"key":"mlstabc609bib24","article-title":"A scalable decoder micro-architecture for fault-tolerant quantum computing","author":"Das","year":"2020"},{"key":"mlstabc609bib25","doi-asserted-by":"publisher","first-page":"602","DOI":"10.1126\/science.aag2302","article-title":"Solving the quantum many-body problem with artificial neural networks","volume":"355","author":"Carleo","year":"2017","journal-title":"Science"},{"key":"mlstabc609bib26","doi-asserted-by":"crossref","DOI":"10.1038\/s41467-018-07520-3","article-title":"Constructing exact representations of quantum many-body systems with deep neural networks","author":"Carleo","year":"2018"},{"key":"mlstabc609bib27","doi-asserted-by":"publisher","first-page":"662","DOI":"10.1038\/s41467-017-00705-2","article-title":"Efficient representation of quantum many-body states with deep neural networks","volume":"8","author":"Gao","year":"2017","journal-title":"Nat. Commun."},{"key":"mlstabc609bib28","doi-asserted-by":"publisher","first-page":"435","DOI":"10.1038\/nphys4037","article-title":"Learning phase transitions by confusion","volume":"13","author":"Van Nieuwenburg","year":"2017","journal-title":"Nat. Phys."},{"key":"mlstabc609bib29","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevB.97.134109","article-title":"Identifying quantum phase transitions with adversarial neural networks","volume":"97","author":"Huembeli","year":"2018","journal-title":"Phys. Rev. B"},{"key":"mlstabc609bib30","doi-asserted-by":"publisher","first-page":"431","DOI":"10.1038\/nphys4035","article-title":"Machine learning phases of matter","volume":"13","author":"Carrasquilla","year":"2017","journal-title":"Nat. Phys."},{"key":"mlstabc609bib31","article-title":"Quantum phase recognition via unsupervised machine learning","author":"Broecker","year":"2017"},{"key":"mlstabc609bib32","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevB.95.245134","article-title":"Probing many-body localization with neural networks","volume":"95","author":"Schindler","year":"2017","journal-title":"Phys. Rev. B"},{"key":"mlstabc609bib33","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1073\/pnas.1714936115","article-title":"Active learning machine learns to create new quantum experiments","author":"Melnikov","year":"2018","journal-title":"Proc. Natl. Acad. Sci."},{"key":"mlstabc609bib34","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevX.8.031084","article-title":"Reinforcement learning with neural networks for quantum feedback","volume":"8","author":"F\u00f6sel","year":"2018","journal-title":"Phys. Rev. X"},{"key":"mlstabc609bib35","doi-asserted-by":"publisher","DOI":"10.1016\/j.physleta.2020.126353","article-title":"Reinforcement learning for optimal error correction of toric codes","volume":"384","author":"Domingo Colomer","year":"2020","journal-title":"Phys. Lett."},{"key":"mlstabc609bib36","doi-asserted-by":"publisher","first-page":"183","DOI":"10.22331\/q-2019-09-02-183","article-title":"Quantum error correction for the toric code using deep reinforcement learning","volume":"3","author":"Andreasson","year":"2019","journal-title":"Quantum"},{"key":"mlstabc609bib37","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevResearch.2.023230","article-title":"Deep q-learning decoder for depolarizing noise on the toric code","volume":"2","author":"Fitzek","year":"2020","journal-title":"Phys. Rev. Res."},{"key":"mlstabc609bib38","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.119.030501","article-title":"Neural decoder for topological codes","volume":"119","author":"Torlai","year":"2017","journal-title":"Phys. Rev. Lett."},{"key":"mlstabc609bib39","doi-asserted-by":"publisher","DOI":"10.1088\/2058-9565\/aa955a","article-title":"Decoding small surface codes with feedforward neural networks","volume":"3","author":"Varsamopoulos","year":"2017","journal-title":"Quantum Sci. Technol."},{"key":"mlstabc609bib40","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-017-11266-1","article-title":"Deep neural network probabilistic decoder for stabilizer codes","volume":"7","author":"Krastanov","year":"2017","journal-title":"Sci. Rep."},{"key":"mlstabc609bib41","article-title":"Advantages of versatile neural-network decoding for topological codes","author":"Maskara","year":"2018"},{"key":"mlstabc609bib42","doi-asserted-by":"publisher","first-page":"68","DOI":"10.22331\/q-2018-05-24-68","article-title":"Scalable neural network decoders for higher dimensional quantum codes","volume":"2","author":"Breuckmann","year":"2018","journal-title":"Quantum"},{"key":"mlstabc609bib43","article-title":"Neural network decoders for large-distance 2D toric codes","author":"Ni","year":"2018"},{"key":"mlstabc609bib44","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s42484-020-00015-9","article-title":"Decoding surface code with a distributed neural network\u2013based decoder","volume":"2","author":"Varsamopoulos","year":"2020","journal-title":"Quantum Machine Intell."},{"key":"mlstabc609bib45","doi-asserted-by":"crossref","DOI":"10.1088\/2058-9565\/aad1f7","article-title":"Deep neural decoders for near term fault-tolerant experiments","author":"Chamberland","year":"2018"},{"key":"mlstabc609bib46","doi-asserted-by":"publisher","first-page":"48","DOI":"10.22331\/q-2018-01-29-48","article-title":"Machine-learning-assisted correction of correlated qubit errors in a topological code","volume":"2","author":"Baireuther","year":"2018","journal-title":"Quantum"},{"key":"mlstabc609bib47","article-title":"Neural network decoder for topological color codes with circuit level noise","author":"Baireuther","year":"2018"},{"key":"mlstabc609bib48","doi-asserted-by":"publisher","first-page":"300","DOI":"10.1109\/TC.2019.2948612","article-title":"Comparing neural network based decoders for the surface code","volume":"69","author":"Varsamopoulos","year":"2019","journal-title":"IEEE Trans. Comput."},{"key":"mlstabc609bib49","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"mlstabc609bib50","article-title":"Prioritized experience replay","author":"Schaul","year":"2015"},{"key":"mlstabc609bib51","first-page":"p 5","volume":"vol 2","author":"Van Hasselt","year":"2016"},{"key":"mlstabc609bib52","article-title":"Dueling network architectures for deep reinforcement learning","author":"Wang","year":"2015"},{"key":"mlstabc609bib53","article-title":"Mastering chess and shogi by self-play with a general reinforcement learning algorithm","author":"Silver","year":"2017"},{"key":"mlstabc609bib54","doi-asserted-by":"publisher","first-page":"484 EP","DOI":"10.1038\/nature16961","article-title":"Mastering the game of go with deep neural networks and tree search","volume":"529","author":"Silver","year":"2016","journal-title":"Nature"},{"key":"mlstabc609bib55","doi-asserted-by":"publisher","first-page":"354","DOI":"10.1038\/nature24270","article-title":"Mastering the game of go without human knowledge","volume":"550","author":"Silver","year":"2017","journal-title":"Nature"},{"key":"mlstabc609bib56","article-title":"Deepq-decoding","author":"Sweke","year":"2018"},{"key":"mlstabc609bib57","article-title":"Stabilizer codes and quantum error correction","author":"Gottesman","year":"1997"},{"key":"mlstabc609bib58","doi-asserted-by":"publisher","DOI":"10.1088\/1367-2630\/16\/6\/063038","article-title":"Fast decoders for qudit topological codes","volume":"16","author":"Anwar","year":"2014","journal-title":"New J. Phys."},{"key":"mlstabc609bib59","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.90.062320","article-title":"Low-distance surface codes under realistic quantum noise","volume":"90","author":"Tomita","year":"2014","journal-title":"Phys. Rev. A"},{"key":"mlstabc609bib60","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.89.022321","article-title":"Fault-tolerant thresholds for quantum error correction with the surface code","volume":"89","author":"Stephens","year":"2014","journal-title":"Phys. Rev. A"},{"key":"mlstabc609bib61","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1023\/A:1011233615437","article-title":"The invariants of the Clifford groups","volume":"24","author":"Nebe","year":"2001","journal-title":"Des. Codes Cryptogr."},{"key":"mlstabc609bib62","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevA.71.022316","article-title":"Universal quantum computation with ideal Clifford gates and noisy ancillas","volume":"71","author":"Bravyi","year":"2005","journal-title":"Phys. Rev. A"},{"key":"mlstabc609bib63","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevX.7.021029","article-title":"Poking holes and cutting corners to achieve Clifford gates with the surface code","volume":"7","author":"Brown","year":"2017","journal-title":"Phys. Rev. X"},{"key":"mlstabc609bib64","doi-asserted-by":"publisher","DOI":"10.1088\/1367-2630\/14\/12\/123011","article-title":"Surface code quantum computing by lattice surgery","volume":"14","author":"Horsman","year":"2012","journal-title":"New J. Phys."},{"key":"mlstabc609bib65","doi-asserted-by":"publisher","first-page":"62","DOI":"10.22331\/q-2018-05-04-62","article-title":"Lattice surgery with a twist: simplifying Clifford gates of surface codes","volume":"2","author":"Litinski","year":"2018","journal-title":"Quantum"},{"key":"mlstabc609bib66","author":"Sutton","year":"2018"},{"key":"mlstabc609bib67","article-title":"Almost-linear time decoding algorithm for topological codes","author":"Delfosse","year":"2017"},{"key":"mlstabc609bib68","author":"Plappert","year":"2016"},{"key":"mlstabc609bib69","first-page":"1928","author":"Mnih","year":"2016"}],"container-title":["Machine Learning: Science and Technology"],"original-title":[],"link":[{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abc609","content-type":"text\/html","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abc609\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abc609","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abc609\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abc609\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abc609","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abc609\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abc609\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abc609\/pdf","content-type":"application\/pdf","content-version":"am","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abc609\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abc609\/pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"},{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abc609","content-type":"text\/html","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,1,21]],"date-time":"2022-01-21T00:56:07Z","timestamp":1642726567000},"score":1,"resource":{"primary":{"URL":"https:\/\/iopscience.iop.org\/article\/10.1088\/2632-2153\/abc609"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,28]]},"references-count":69,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2020,12,28]]},"published-print":{"date-parts":[[2021,6,1]]}},"URL":"https:\/\/doi.org\/10.1088\/2632-2153\/abc609","relation":{},"ISSN":["2632-2153"],"issn-type":[{"value":"2632-2153","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12,28]]},"assertion":[{"value":"Reinforcement learning decoders for fault-tolerant quantum computation","name":"article_title","label":"Article Title"},{"value":"Machine Learning: Science and Technology","name":"journal_title","label":"Journal Title"},{"value":"paper","name":"article_type","label":"Article Type"},{"value":"\u00a9 2020 The Author(s). Published by IOP Publishing Ltd","name":"copyright_information","label":"Copyright Information"},{"value":"2020-07-27","name":"date_received","label":"Date Received","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2020-10-29","name":"date_accepted","label":"Date Accepted","group":{"name":"publication_dates","label":"Publication dates"}},{"value":"2020-12-28","name":"date_epub","label":"Online publication date","group":{"name":"publication_dates","label":"Publication dates"}}]}}