{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,18]],"date-time":"2026-07-18T02:40:31Z","timestamp":1784342431506,"version":"3.55.0"},"reference-count":76,"publisher":"Association for Computing Machinery (ACM)","issue":"OOPSLA2","funder":[{"DOI":"10.13039\/501100000266","name":"EPSRC","doi-asserted-by":"crossref","award":["EP\/Y005244\/1, EP\/W032635\/1"],"award-info":[{"award-number":["EP\/Y005244\/1, EP\/W032635\/1"]}],"id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Program. Lang."],"published-print":{"date-parts":[[2025,10,9]]},"abstract":"<jats:p>Quantum computing platforms rely on simulators for modelling circuit behaviour prior to hardware execution, where inconsistencies can lead to costly errors. While existing formal validation methods typically target specific compiler components to manage state explosion, they often miss critical bugs. Meanwhile, conventional testing lacks systematic exploration of corner cases and realistic execution scenarios, resulting in both false positives and negatives.<\/jats:p>\n                  <jats:p>We present FuzzQ, a novel framework that bridges this gap by combining formal methods with structured test generation and fuzzing for quantum simulators. Our approach employs differential benchmarking complemented by mutation testing and invariant checking. At its core, FuzzQ utilises our Alloy-based formal model of QASM 3.0, which encodes the semantics of quantum circuits to enable automated analysis and to generate structurally diverse, constraint-guided quantum circuits with guaranteed properties. We introduce several test oracles to assess both Alloy\u2019s modelling of QASM 3.0 and simulator correctness, including invariant-based checks, statistical distribution tests, and a novel cross-simulator unitary consistency check that verifies functional equivalence modulo global phase, revealing discrepancies that standard statevector comparisons fail to detect in cross-platform differential testing.<\/jats:p>\n                  <jats:p>We evaluate FuzzQ on both Qiskit and Cirq, demonstrating its platform-agnostic effectiveness. By executing over 800,000 quantum circuits to completion, we assess throughput, code and circuit coverage, and simulator performance metrics, including sensitivity, correctness, and memory overhead. Our analysis revealed eight simulator bugs, six previously undocumented. We also outline a path for extending the framework to support mixed-state simulations under realistic noise models.<\/jats:p>","DOI":"10.1145\/3763100","type":"journal-article","created":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T08:49:50Z","timestamp":1759999790000},"page":"1400-1428","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Shaking Up Quantum Simulators with Fuzzing and Rigour"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3173-8636","authenticated-orcid":false,"given":"Vasileios","family":"Klimis","sequence":"first","affiliation":[{"name":"Queen Mary University of London, London, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-3285-9468","authenticated-orcid":false,"given":"Avner","family":"Bensoussan","sequence":"additional","affiliation":[{"name":"King\u2019s College London, London, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2857-5570","authenticated-orcid":false,"given":"Elena","family":"Chachkarova","sequence":"additional","affiliation":[{"name":"King\u2019s College London, London, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3099-1189","authenticated-orcid":false,"given":"Karine","family":"Even-Mendoza","sequence":"additional","affiliation":[{"name":"King\u2019s College London, London, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9687-8587","authenticated-orcid":false,"given":"Sophie","family":"Fortz","sequence":"additional","affiliation":[{"name":"King\u2019s College London, London, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1885-2941","authenticated-orcid":false,"given":"Connor","family":"Lenihan","sequence":"additional","affiliation":[{"name":"King\u2019s College London, London, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,10,9]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3528230.3529189"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2022.08.003"},{"key":"e_1_3_2_4_2","unstructured":"Amazon Web Services (AWS). Accessed: 2025-01-17. Amazon Braket. https:\/\/aws.amazon.com\/braket\/."},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2014.2372785"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-024-08148-8"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1103\/RevModPhys.94.015004"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-64573-0_6"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1103\/RevModPhys.86.419"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1103\/RevModPhys.95.045005"},{"key":"e_1_3_2_11_2","author":"Campos Jos\u00e9","year":"2021","unstructured":"Jos\u00e9 Campos and Andr\u00e9 Souto. 2021. QBugs: A Collection of Reproducible Bugs in Quantum Algorithms and a Supporting Infrastructure to Enable Controlled Quantum Software Testing and Debugging Experiments. http:\/\/arxiv.org\/abs\/2103.16968 arXiv:2103.16968 [cs].","journal-title":"QBugs: A Collection of Reproducible Bugs in Quantum Algorithms and a Supporting Infrastructure to Enable Controlled Quantum Software Testing and Debugging Experiments"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/3192366.3192373"},{"key":"e_1_3_2_13_2","unstructured":"Cirq GitHub Issue Tracker. 2024. GitHub Issue #6810. https:\/\/github.com\/quantumlib\/Cirq\/issues\/6810. GitHub accessed August 2025."},{"key":"e_1_3_2_14_2","unstructured":"Cirq GitHub Issue Tracker. 2024. GitHub Issue #6811. https:\/\/github.com\/quantumlib\/Cirq\/issues\/6811. GitHub accessed August 2025."},{"key":"e_1_3_2_15_2","unstructured":"Fuzz4All Contributors. 2024. Qiskit Target Runner for Fuzz4All. https:\/\/github.com\/fuzz4all\/fuzz4all\/blob\/main\/ Fuzz4All\/target\/QISKIT\/QISKIT.py. Accessed: 2025-07-21."},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3505636"},{"key":"e_1_3_2_17_2","unstructured":"D-Wave Systems. Accessed Jan. 2025. D-Wave Documentation. https:\/\/docs.ocean.dwavesys.com\/en\/stable\/."},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","unstructured":"Cirq Developers. 2024. Cirq. doi:10.5281\/zenodo.11398048","DOI":"10.5281\/zenodo.11398048"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevResearch.5.043210"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.5555\/3358807.3358809"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510454.3528649"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1002\/smr.2419"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1002\/spe.3039"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1088\/0034-4885\/75\/8\/082401"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/3434318"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3307650.3322213"},{"key":"e_1_3_2_27_2","unstructured":"IBM Quantum Computing. Accessed: 2025-01-17. Qiskit. https:\/\/www.ibm.com\/quantum\/qiskit."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3338843"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524482.3527651"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3689716"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3691621.3694959"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.16918102"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3571253"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3550488"},{"key":"e_1_3_2_35_2","unstructured":"Gushu Li Li Zhou Nengkun Yu Yufei Ding Mingsheng Ying and Yuan Xie. 2020. Proq: Projection-based Runtime Assertions for Debugging on a Quantum Computer. http:\/\/arxiv.org\/abs\/1911.12855 arXiv:1911.12855 [quant-ph]."},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3563309"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/IC3IT63743.2024.10869429"},{"key":"e_1_3_2_38_2","unstructured":"Junjie Luo Pengzhan Zhao Zhongtao Miao Shuhan Lan and Jianjun Zhao. 2022. A Comprehensive Study of Bug Fixes in Quantum Programs. https:\/\/www.computer.org\/csdl\/proceedings-article\/saner\/2022\/378600b239\/1FbT6n3hGaA"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-37057-1_22"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE51524.2021.9678563"},{"key":"e_1_3_2_41_2","doi-asserted-by":"crossref","unstructured":"Sara Ayman Metwalli and Rodney Van Meter. 2022. A Tool For Debugging Quantum Circuits. https:\/\/www.computer.org\/csdl\/proceedings-article\/qce\/2022\/911300a624\/1IvLZRlty80","DOI":"10.1109\/QCE53715.2022.00085"},{"key":"e_1_3_2_42_2","unstructured":"Microsoft. Accessed: 2025-01-17. Microsoft Azure Quantum. https:\/\/azure.microsoft.com\/en-us\/products\/quantum\/."},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-NIER.2019.00023"},{"key":"e_1_3_2_44_2","unstructured":"Michael A. Nielsen and Isaac L. Chuang. 2010. Quantum Computation and Quantum Information (10th Anniversary edition). Cambridge University Press Cambridge."},{"key":"e_1_3_2_45_2","unstructured":"NumPy Developers. 2018. Einsum symbol limitations. https:\/\/github.com\/numpy\/numpy\/issues\/11938. GitHub issue #11938 accessed July 2025."},{"key":"e_1_3_2_46_2","unstructured":"NumPy Developers. 2022. How is NumPy's einsum so much slower than other libraries? https:\/\/github.com\/numpy\/numpy\/issues\/22604. GitHub issue #22604 accessed July 2025."},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/3527330"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00202"},{"key":"e_1_3_2_49_2","first-page":"1","article-title":"A Survey on Testing and Analysis of Quantum Software","author":"Paltenghi Matteo","year":"2024","unstructured":"Matteo Paltenghi and Michael Pradel. 2024. A Survey on Testing and Analysis of Quantum Software. ArXiv abs\/2410.00650 (2024), 1\u201310. https:\/\/api.semanticscholar.org\/CorpusID:273023238","journal-title":"ArXiv abs\/2410.00650 (2024)"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","unstructured":"John Preskill. 2018. Quantum Computing in the NISQ era and beyond. Quantum 2 (Aug. 2018) 79. doi:10.1145\/237814.237866","DOI":"10.1145\/237814.237866"},{"key":"e_1_3_2_51_2","unstructured":"QIR Alliance. Accessed: 2025-01-03. QIR Alliance Projects. https:\/\/www.qir-alliance.org\/projects\/"},{"key":"e_1_3_2_52_2","unstructured":"Qiskit GitHub Issue Tracker. 2024. 0GitHub Issue #13536. https:\/\/github.com\/Qiskit\/qiskit\/issues\/13536. GitHub accessed August 2025."},{"key":"e_1_3_2_53_2","unstructured":"Qiskit GitHub Issue Tracker. 2024. GitHub Issue #13583. https:\/\/github.com\/Qiskit\/qiskit\/issues\/13583. GitHub accessed August 2025."},{"key":"e_1_3_2_54_2","unstructured":"Qiskit GitHub Issue Tracker. 2024. GitHub Issue #2274. https:\/\/github.com\/Qiskit\/qiskit-aer\/issues\/2274. GitHub accessed August 2025."},{"key":"e_1_3_2_55_2","unstructured":"Qiskit GitHub Issue Tracker. 2024. GitHub Issue #2276. https:\/\/github.com\/Qiskit\/qiskit-aer\/issues\/2276. GitHub accessed August 2025."},{"key":"e_1_3_2_56_2","unstructured":"Qiskit GitHub Issue Tracker. 2024. GitHub Issue #4159 and #7120. https:\/\/github.com\/Qiskit\/qiskit\/issues\/4159 https:\/\/github.com\/Qiskit\/qiskit\/issues\/7120. Known bugs GitHub accessed August 2025."},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3371079"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3360561"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/3715106"},{"key":"e_1_3_2_60_2","unstructured":"Rigetti Computing. Accessed: 2025-01-17. Rigetti Forest SDK. https:\/\/www.rigetti.com\/forest."},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/1449764.1449803"},{"key":"e_1_3_2_62_2","unstructured":"Hinrich Sch\u00fctze and Christopher D. Manning. 1999. Foundations of Statistical Natural Language Processing. MIT Press Cambridge MA. 304 pages."},{"key":"e_1_3_2_63_2","unstructured":"George W. Snedecor and William G. Cochran. 1989. Statistical Methods (8th ed.). Iowa State University Press Ames Iowa."},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICST.2017.31"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/3519939.3523431"},{"key":"e_1_3_2_66_2","unstructured":"Mypy Team. Accessed: 2025-03-20. Mypy: A static type checker for Python. https:\/\/github.com\/python\/mypy."},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE51524.2021.9678792"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236024.3264594"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1145\/3695053.3730992"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510454.3516839"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/3512290.3528869"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1145\/3009837.3009838"},{"key":"e_1_3_2_73_2","unstructured":"Xanadu Quantum Technologies. Accessed: 2025-01-17. PennyLane. https:\/\/pennylane.ai\/."},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639121"},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1145\/3527316"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","unstructured":"Pengzhan Zhao Jianjun Zhao and Lei Ma. 2021. Identifying Bug Patterns in Quantum Programs. 16\u201321 pages. doi:10.1109\/Q-SE52541.2021.00011","DOI":"10.1109\/Q-SE52541.2021.00011"},{"key":"e_1_3_2_77_2","doi-asserted-by":"crossref","unstructured":"Pengzhan Zhao Jianjun Zhao Zhongtao Miao and Shuhan Lan. 2021. Bugs4Q: A Benchmark of Real Bugs for Quantum Programs. http:\/\/arxiv.org\/abs\/2108.09744 arXiv:2108.09744 [cs].","DOI":"10.1109\/ASE51524.2021.9678908"}],"container-title":["Proceedings of the ACM on Programming Languages"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3763100","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T10:03:27Z","timestamp":1784196207000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3763100"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,9]]},"references-count":76,"journal-issue":{"issue":"OOPSLA2","published-print":{"date-parts":[[2025,10,9]]}},"alternative-id":["10.1145\/3763100"],"URL":"https:\/\/doi.org\/10.1145\/3763100","relation":{},"ISSN":["2475-1421"],"issn-type":[{"value":"2475-1421","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,9]]},"assertion":[{"value":"2025-03-22","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-08-12","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-10-09","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}