{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,19]],"date-time":"2025-09-19T09:31:37Z","timestamp":1758274297453,"version":"3.40.3"},"publisher-location":"Cham","reference-count":91,"publisher":"Springer Nature Switzerland","isbn-type":[{"type":"print","value":"9783031641350"},{"type":"electronic","value":"9783031641367"}],"license":[{"start":{"date-parts":[[2024,1,1]],"date-time":"2024-01-01T00:00:00Z","timestamp":1704067200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,8,23]],"date-time":"2024-08-23T00:00:00Z","timestamp":1724371200000},"content-version":"vor","delay-in-days":235,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100012163","name":"Universit\u00e4t Innsbruck","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100012163","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Quantum software\u2014like classic software\u2014needs to be designed, specified, developed, and, most importantly, tested by developers. Writing tests is a complex, error-prone, and time-consuming task. Due to the particular properties of quantum physics (e.g., superposition), quantum software is inherently more complex to develop and effectively test than classical software. Nevertheless, some preliminary works have tried to bring commonly used classical testing practices for quantum computing to assess and improve the quality of quantum programs. In this chapter, we first gather 16 quantum software testing techniques that have been proposed for the IBM quantum framework, Qiskit. Then, whenever possible, we illustrate the usage of each technique (through the proposed tool that implements it, if available) on a given running example. We showcase that although several works have been proposed to ease the burn of testing quantum software, we are still in the early stages of testing in the quantum world. Researchers should focus on delivering artifacts that are usable without much hindrance to the rest of the community, and the development of quantum benchmarks should be a priority to facilitate reproducibility, replicability, and comparison between different testing techniques.<\/jats:p>","DOI":"10.1007\/978-3-031-64136-7_5","type":"book-chapter","created":{"date-parts":[[2024,8,23]],"date-time":"2024-08-23T13:48:28Z","timestamp":1724420908000},"page":"93-123","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Verification and Validation of Quantum Software"],"prefix":"10.1007","author":[{"given":"Daniel","family":"Fortunato","sequence":"first","affiliation":[]},{"given":"Luis","family":"Jim\u00e9nez-Navajas","sequence":"additional","affiliation":[]},{"given":"Jos\u00e9","family":"Campos","sequence":"additional","affiliation":[]},{"given":"Rui","family":"Abreu","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,8,23]]},"reference":[{"key":"5_CR1","unstructured":"Russell, J.: IBM Quantum Update: Q System One Launch, New Collaborators, and QC Center Plans. HPC Wire"},{"key":"5_CR2","unstructured":"Collins, H., Nay, C.: IBM Unveils 400 Qubit-Plus Quantum Processor and Next-Generation IBM Quantum. IBM Newsroom"},{"key":"5_CR3","unstructured":"Ferreira, F.: An Exploratory Study on the Usage of Quantum Programming Languages. Available at http:\/\/hdl.handle.net\/10451\/56751"},{"issue":"5","key":"5_CR4","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1109\/MS.2021.3087755","volume":"38","author":"JL Hevia","year":"2021","unstructured":"Hevia, J.L., Peterssen, G., Ebert, C., Piattini, M.: Quantum computing. IEEE Software 38(5), 7\u201315 (2021). https:\/\/doi.org\/10.1109\/MS.2021.3087755","journal-title":"IEEE Software"},{"issue":"4","key":"5_CR5","doi-asserted-by":"publisher","first-page":"2419","DOI":"10.1002\/smr.2419","volume":"35","author":"A Barrera","year":"2023","unstructured":"Barrera, A., Guzm\u00e1n, I., Polo, M., Piattini, M.: Quantum software testing: state of the art. J. Software Evol. Process 35(4), 2419 (2023). https:\/\/doi.org\/10.1002\/smr.2419","journal-title":"J. Software Evol. Process"},{"key":"5_CR6","doi-asserted-by":"crossref","unstructured":"Weder, B., Barzen, J., Leymann, F., Salm, M., Vietz, D.: The Quantum Software Lifecycle. In: Proceedings of the 1st ACM SIGSOFT International Workshop on Architectures and Paradigms for Engineering Quantum Software. APEQS 2020, pp. 2\u20139. Association for Computing Machinery, New York, NY, USA (2020). https:\/\/doi.org\/10.1145\/3412451.3428497","DOI":"10.1145\/3412451.3428497"},{"key":"5_CR7","doi-asserted-by":"crossref","unstructured":"Arias, D., Garc\u00eda Rodr\u00edguez de Guzm\u00e1n, I., Rodr\u00edguez, M., Terres, E.B., Sanz, B., Gaviria de la Puerta, J., Pastor, I., Zubillaga, A., Garc\u00eda Bringas, P.: Let\u2019s do it right the first time: Survey on security concerns in the way to quantum software engineering. Neurocomputing 538, 126199 (2023). https:\/\/doi.org\/10.1016\/j.neucom.2023.03.060","DOI":"10.1016\/j.neucom.2023.03.060"},{"key":"5_CR8","unstructured":"Tao Yue, P.A., Ali, S.: Quantum Software Testing: Challenges, Early Achievements, and Opportunities. ERCIM News"},{"key":"5_CR9","doi-asserted-by":"crossref","unstructured":"Ying, M.: Floyd\u2013Hoare logic for quantum programs. ACM Trans. Program. Lang. Syst. 33(6) (2012). https:\/\/doi.org\/10.1145\/2049706.2049708","DOI":"10.1145\/2049706.2049708"},{"key":"5_CR10","doi-asserted-by":"crossref","unstructured":"Zhou, L., Yu, N., Ying, M.: An Applied Quantum Hoare Logic. In: Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation. PLDI 2019, pp. 1149\u20131162. Association for Computing Machinery, New York, NY, USA (2019). https:\/\/doi.org\/10.1145\/3314221.3314584","DOI":"10.1145\/3314221.3314584"},{"key":"5_CR11","doi-asserted-by":"crossref","unstructured":"Honarvar, S., Mousavi, M.R., Nagarajan, R.: Property-Based Testing of Quantum Programs in q#. In: Proceedings of the IEEE\/ACM 42nd International Conference on Software Engineering Workshops, pp. 430\u2013435 (2020)","DOI":"10.1145\/3387940.3391459"},{"key":"5_CR12","doi-asserted-by":"crossref","unstructured":"Zhao, P., Wu, X., Li, Z., Zhao, J.: QChecker: Detecting Bugs in Quantum Programs via Static Analysis (2023)","DOI":"10.1109\/Q-SE59154.2023.00014"},{"key":"5_CR13","doi-asserted-by":"crossref","unstructured":"Yu, N., Palsberg, J.: Quantum Abstract Interpretation. In: Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation, pp. 542\u2013558. ACM, Virtual Canada (2021). https:\/\/doi.org\/10.1145\/3453483.3454061. https:\/\/dl.acm.org\/doi\/10.1145\/3453483.3454061","DOI":"10.1145\/3453483.3454061"},{"key":"5_CR14","doi-asserted-by":"publisher","unstructured":"Xia, S., Zhao, J.: Static Entanglement Analysis of Quantum Programs (2023). https:\/\/doi.org\/10.48550\/arXiv.2304.05049. arXiv:2304.05049 [quant-ph]","DOI":"10.48550\/arXiv.2304.05049"},{"key":"5_CR15","doi-asserted-by":"publisher","unstructured":"Kaul, M., K\u00fcchler, A., Banse, C.: A Uniform Representation of Classical and Quantum Source Code for Static Code Analysis (2023). https:\/\/doi.org\/10.48550\/arXiv.2308.06113. arXiv:2308.06113 [cs]","DOI":"10.48550\/arXiv.2308.06113"},{"key":"5_CR16","unstructured":"Wang, J., Gao, M., Jiang, Y., Lou, J., Gao, Y., Zhang, D., Sun, J.: QuanFuzz: Fuzz Testing of Quantum Program (2018). arXiv:1810.10310 [cs]"},{"key":"5_CR17","doi-asserted-by":"publisher","unstructured":"Wang, J., Zhang, Q., Xu, G.H., Kim, M.: QDiff: Differential Testing of Quantum Software Stacks. In: 2021 36th IEEE\/ACM International Conference on Automated Software Engineering (ASE), pp. 692\u2013704 (2021). https:\/\/doi.org\/10.1109\/ASE51524.2021.9678792","DOI":"10.1109\/ASE51524.2021.9678792"},{"key":"5_CR18","doi-asserted-by":"crossref","unstructured":"Wang, X., Yu, T., Arcaini, P., Yue, T., Ali, S.: Mutation-Based Test Generation for Quantum Programs with Multi-Objective Search. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 1345\u20131353. ACM, Boston Massachusetts (2022). https:\/\/doi.org\/10.1145\/3512290.3528869. https:\/\/dl.acm.org\/doi\/10.1145\/3512290.3528869","DOI":"10.1145\/3512290.3528869"},{"key":"5_CR19","doi-asserted-by":"crossref","unstructured":"Abreu, R., Fernandes, J.P., Llana, L., Tavares, G.: Metamorphic Testing of Oracle Quantum Programs. In: Proceedings of the 3rd International Workshop on Quantum Software Engineering, pp. 16\u201323. ACM, Pittsburgh Pennsylvania (2022). https:\/\/doi.org\/10.1145\/3528230.3529189. https:\/\/dl.acm.org\/doi\/10.1145\/3528230.3529189","DOI":"10.1145\/3528230.3529189"},{"key":"5_CR20","doi-asserted-by":"publisher","unstructured":"Paltenghi, M., Pradel, M.: MorphQ: Metamorphic Testing of Quantum Computing Platforms (2022). https:\/\/doi.org\/10.48550\/arXiv.2206.01111. arXiv:2206.01111 [cs]","DOI":"10.48550\/arXiv.2206.01111"},{"key":"5_CR21","doi-asserted-by":"publisher","unstructured":"Wang, X., Arcaini, P., Yue, T., Ali, S.: Application of Combinatorial Testing to Quantum Programs. In: 2021 IEEE 21st International Conference on Software Quality, Reliability and Security (QRS), pp. 179\u2013188 (2021). https:\/\/doi.org\/10.1109\/QRS54544.2021.00029","DOI":"10.1109\/QRS54544.2021.00029"},{"key":"5_CR22","doi-asserted-by":"crossref","unstructured":"Wang, X., Arcaini, P., Yue, T., Ali, S.: QuCAT: A Combinatorial Testing Tool for Quantum Software (2023). https:\/\/arxiv.org\/abs\/2309.00119v1","DOI":"10.1109\/ASE56229.2023.00062"},{"key":"5_CR23","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1007\/978-3-030-88106-1_2","volume-title":"Search-Based Software Engineering. Lecture Notes in Computer Science","author":"X Wang","year":"2021","unstructured":"Wang, X., Arcaini, P., Yue, T., Ali, S.: Generating Failing Test Suites for Quantum Programs With Search. In: O\u2019Reilly, U.-M., Devroey, X. (eds.) Search-Based Software Engineering. Lecture Notes in Computer Science, pp. 9\u201325. Springer, Cham (2021). https:\/\/doi.org\/10.1007\/978-3-030-88106-1_2"},{"key":"5_CR24","doi-asserted-by":"crossref","unstructured":"Wang, X., Arcaini, P., Yue, T., Ali, S.: QuSBT: Search-Based Testing of Quantum Programs. In: Proceedings of the ACM\/IEEE 44th International Conference on Software Engineering: Companion Proceedings, pp. 173\u2013177 (2022)","DOI":"10.1109\/ICSE-Companion55297.2022.9793826"},{"key":"5_CR25","doi-asserted-by":"publisher","unstructured":"Ali, S., Arcaini, P., Wang, X., Yue, T.: Assessing the Effectiveness of Input and Output Coverage Criteria for Testing Quantum Programs. In: 2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST), pp. 13\u201323 (2021). https:\/\/doi.org\/10.1109\/ICST49551.2021.00014","DOI":"10.1109\/ICST49551.2021.00014"},{"key":"5_CR26","doi-asserted-by":"publisher","unstructured":"Wang, X., Arcaini, P., Yue, T., Ali, S.: Quito: A Coverage-Guided Test Generator for Quantum Programs. In: 2021 36th IEEE\/ACM International Conference on Automated Software Engineering (ASE), pp. 1237\u20131241 (2021). https:\/\/doi.org\/10.1109\/ASE51524.2021.9678798","DOI":"10.1109\/ASE51524.2021.9678798"},{"key":"5_CR27","doi-asserted-by":"publisher","unstructured":"Rajak, A., Suzuki, S., Dutta, A., Chakrabarti, B.K.: Quantum annealing: an overview. Phil. Trans. Roy. Soc. A Math. Phys. Eng. Sci. 381(2241), 20210417 (2023). https:\/\/doi.org\/10.1098\/rsta.2021.0417. https:\/\/royalsocietypublishing.org\/doi\/pdf\/10.1098\/rsta.2021.0417","DOI":"10.1098\/rsta.2021.0417"},{"key":"5_CR28","doi-asserted-by":"publisher","unstructured":"Aleksandrowicz, G., Alexander, T., Barkoutsos, P., Bello, L., Ben-Haim, Y., Bucher, D., Cabrera-Hern\u00e1ndez, F.J., Carballo-Franquis, J., Chen, A., Chen, C.-F., Chow, J.M., C\u00f3rcoles-Gonzales, A.D., Cross, A.J., Cross, A., Cruz-Benito, J., Culver, C., Gonz\u00e1lez, S.D.L.P., Torre, E.D.L., Ding, D., Dumitrescu, E., Duran, I., Eendebak, P., Everitt, M., Sertage, I.F., Frisch, A., Fuhrer, A., Gambetta, J., Gago, B.G., Gomez-Mosquera, J., Greenberg, D., Hamamura, I., Havlicek, V., Hellmers, J., Herok, Horii, H., Hu, S., Imamichi, T., Itoko, T., Javadi-Abhari, A., Kanazawa, N., Karazeev, A., Krsulich, K., Liu, P., Luh, Y., Maeng, Y., Marques, M., Mart\u00edn-Fern\u00e1ndez, F.J., McClure, D.T., McKay, D., Meesala, S., Mezzacapo, A., Moll, N., Rodr\u00edguez, D.M., Nannicini, G., Nation, P., Ollitrault, P., O\u2019Riordan, L.J., Paik, H., P\u00e9rez, J., Phan, A., Pistoia, M., Prutyanov, V., Reuter, M., Rice, J., Davila, A.R., Rudy, R.H.P., Ryu, M., Sathaye, N., Schnabel, C., Schoute, E., Setia, K., Shi, Y., Silva, A., Siraichi, Y., Sivarajah, S., Smolin, J.A., Soeken, M., Takahashi, H., Tavernelli, I., Taylor, C., Taylour, P., Trabing, K., Treinish, M., Turner, W., Vogt-Lee, D., Vuillot, C., Wildstrom, J.A., Wilson, J., Winston, E., Wood, C., Wood, S., W\u00f6rner, S., Akhalwaya, I.Y., Zoufal, C.: Qiskit: An Open-source Framework for Quantum Computing. Zenodo (2019). https:\/\/doi.org\/10.5281\/zenodo.2562111","DOI":"10.5281\/zenodo.2562111"},{"key":"5_CR29","doi-asserted-by":"publisher","unstructured":"Nielsen, M.A., Chuang, I.L.: Quantum Computation and Quantum Information: 10th Anniversary Edition. Cambridge University Press, Cambridge (2010). https:\/\/doi.org\/10.1017\/CBO9780511976667","DOI":"10.1017\/CBO9780511976667"},{"key":"5_CR30","unstructured":"Alaqail, H., Ahmed, S.: Overview of software testing standard iso\/iec\/ieee 29119. Int. J. Comput. Sci. Network Secur. (IJCSNS) 18(2), 112\u2013116 (2018)"},{"key":"5_CR31","doi-asserted-by":"publisher","unstructured":"Miranskyy, A., Zhang, L.: On Testing Quantum Programs. In: 2019 IEEE\/ACM 41st International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER), pp. 57\u201360 (2019). https:\/\/doi.org\/10.1109\/ICSE-NIER.2019.00023. http:\/\/arxiv.org\/abs\/1812.09261","DOI":"10.1109\/ICSE-NIER.2019.00023"},{"key":"5_CR32","doi-asserted-by":"publisher","first-page":"111326","DOI":"10.1016\/j.jss.2022.111326","volume":"190","author":"M De Stefano","year":"2022","unstructured":"De Stefano, M., Pecorelli, F., Di Nucci, D., Palomba, F., De Lucia, A.: Software engineering for quantum programming: How far are we? J. Syst. Software 190, 111326 (2022). https:\/\/doi.org\/10.1016\/j.jss.2022.111326","journal-title":"J. Syst. Software"},{"key":"5_CR33","doi-asserted-by":"crossref","unstructured":"Paltenghi, M., Pradel, M.: LintQ: A Static Analysis Framework for Qiskit Quantum Programs (2023). arXiv:2310.00718 [cs]","DOI":"10.1145\/3660802"},{"key":"5_CR34","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TQE.2022.3195061","volume":"3","author":"D Fortunato","year":"2022","unstructured":"Fortunato, D., Campos, J., Abreu, R.: Mutation testing of quantum programs: a case study with Qiskit. IEEE Trans. Quant. Eng. 3, 1\u201317 (2022). https:\/\/doi.org\/10.1109\/TQE.2022.3195061","journal-title":"IEEE Trans. Quant. Eng."},{"key":"5_CR35","doi-asserted-by":"crossref","unstructured":"Fortunato, D., Campos, J., Abreu, R.: Mutation Testing of Quantum Programs Written in QISKit. In: 2022 IEEE\/ACM 44th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion), pp. 358\u2013359 (2022). https:\/\/doi.org\/10.1145\/3510454.3528649","DOI":"10.1109\/ICSE-Companion55297.2022.9793776"},{"key":"5_CR36","doi-asserted-by":"crossref","unstructured":"Fortunato, D., Campos, J., Abreu, R.: QMutPy: A Mutation Testing tool for Quantum algorithms and Applications in Qiskit. In: Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 797\u2013800. ACM, Virtual South Korea (2022). https:\/\/doi.org\/10.1145\/3533767.3543296 . https:\/\/dl.acm.org\/doi\/10.1145\/3533767.3543296","DOI":"10.1145\/3533767.3543296"},{"key":"5_CR37","doi-asserted-by":"publisher","unstructured":"Mendiluze, E., Ali, S., Arcaini, P., Yue, T.: Muskit: A Mutation Analysis Tool for Quantum Software Testing. In: 2021 36th IEEE\/ACM International Conference on Automated Software Engineering (ASE), pp. 1266\u20131270 (2021). https:\/\/doi.org\/10.1109\/ASE51524.2021.9678563","DOI":"10.1109\/ASE51524.2021.9678563"},{"key":"5_CR38","unstructured":"Li, G., Zhou, L., Yu, N., Ding, Y., Ying, M., Xie, Y.: Projection-based runtime assertions for testing and debugging quantum programs (2020). Accepted: 2021-03-14T22:46:19Z"},{"key":"5_CR39","unstructured":"Muqeet, A., Yue, T., Ali, S., Arcaini, P.: Noise-Aware Quantum Software Testing (2023)"},{"key":"5_CR40","doi-asserted-by":"crossref","unstructured":"Zhang, L., Radnejad, M., Miranskyy, A.: Identifying Flakiness in Quantum Programs. Preprint (2023). arXiv:2302.03256","DOI":"10.1109\/ESEM56168.2023.10304850"},{"key":"5_CR41","doi-asserted-by":"crossref","unstructured":"Long, P., Zhao, J.: Testing multi-subroutine quantum programs: From unit testing to integration testing (2023). arXiv:2306.17407 [quant-ph]","DOI":"10.1145\/3656339"},{"key":"5_CR42","unstructured":"Long, P., Zhao, J.: Testing quantum programs with multiple subroutines (2023). arXiv:2208.09206 [cs]"},{"issue":"10","key":"5_CR43","doi-asserted-by":"publisher","first-page":"576","DOI":"10.1145\/363235.363259","volume":"12","author":"CAR Hoare","year":"1969","unstructured":"Hoare, C.A.R.: An axiomatic basis for computer programming. Commun. ACM 12(10), 576\u2013580 (1969). https:\/\/doi.org\/10.1145\/363235.363259","journal-title":"Commun. ACM"},{"key":"5_CR44","doi-asserted-by":"crossref","unstructured":"Yamaguchi, F., Golde, N., Arp, D., Rieck, K.: Modeling and Discovering Vulnerabilities with Code Property Graphs. In: 2014 IEEE Symposium on Security and Privacy, pp. 590\u2013604 (2014). https:\/\/doi.org\/10.1109\/SP.2014.44","DOI":"10.1109\/SP.2014.44"},{"key":"5_CR45","doi-asserted-by":"crossref","unstructured":"Barrera, A.G., Guzm\u00e1n, I.G.-R., Polo, M., Cruz-Lemus, J.A.: In: Serrano, M.A., P\u00e9rez-Castillo, R., Piattini, M. (eds.) Quantum Software Testing: Current Trends and Emerging Proposals, pp. 167\u2013191. Springer, Cham (2022). https:\/\/doi.org\/10.1007\/978-3-031-05324-5_9","DOI":"10.1007\/978-3-031-05324-5_9"},{"issue":"1","key":"5_CR46","doi-asserted-by":"publisher","DOI":"10.1088\/1367-2630\/11\/1\/013006","volume":"11","author":"D Sych","year":"2009","unstructured":"Sych, D., Leuchs, G.: A complete basis of generalized bell states. New J. Phys. 11(1), 013006 (2009). https:\/\/doi.org\/10.1088\/1367-2630\/11\/1\/013006","journal-title":"New J. Phys."},{"key":"5_CR47","doi-asserted-by":"publisher","unstructured":"Harrow, A.W., Hassidim, A., Lloyd, S.: Quantum algorithm for linear systems of equations. Phys. Rev. Lett. 103(15), (2009). https:\/\/doi.org\/10.1103\/physrevlett.103.150502","DOI":"10.1103\/physrevlett.103.150502"},{"issue":"9","key":"5_CR48","doi-asserted-by":"publisher","first-page":"631","DOI":"10.1038\/nphys3029","volume":"10","author":"S Lloyd","year":"2014","unstructured":"Lloyd, S., Mohseni, M., Rebentrost, P.: Quantum principal component analysis. Nature Phys. 10(9), 631\u2013633 (2014) https:\/\/doi.org\/10.1038\/nphys3029","journal-title":"Nature Phys."},{"key":"5_CR49","doi-asserted-by":"crossref","unstructured":"Zhao, P., Zhao, J., Ma, L.: Identifying Bug Patterns in Quantum Programs. In: 2021 IEEE\/ACM 2nd International Workshop on Quantum Software Engineering (Q-SE), pp. 16\u201321. IEEE, Madrid, Spain (2021). https:\/\/doi.org\/10.1109\/Q-SE52541.2021.00011. https:\/\/ieeexplore.ieee.org\/document\/9474564\/","DOI":"10.1109\/Q-SE52541.2021.00011"},{"key":"5_CR50","doi-asserted-by":"publisher","unstructured":"Zhao, P., Zhao, J., Miao, Z., Lan, S.: Bugs4Q: A Benchmark of Real Bugs for Quantum Programs. In: 2021 36th IEEE\/ACM International Conference on Automated Software Engineering (ASE), pp. 1373\u20131376 (2021). https:\/\/doi.org\/10.1109\/ASE51524.2021.9678908","DOI":"10.1109\/ASE51524.2021.9678908"},{"key":"5_CR51","doi-asserted-by":"publisher","unstructured":"Chen, Q., C\u00e2mara, R., Campos, J., Souto, A., Ahmed, I.: The Smelly Eight: An Empirical Study on the Prevalence of Code Smells in Quantum Computing. In: 2023 IEEE\/ACM 45th International Conference on Software Engineering (ICSE), pp. 358\u2013370 (2023). https:\/\/doi.org\/10.1109\/ICSE48619.2023.00041","DOI":"10.1109\/ICSE48619.2023.00041"},{"key":"5_CR52","unstructured":"Cross, A.W., Bishop, L.S., Smolin, J.A., Gambetta, J.M.: Open Quantum Assembly Language (2017)"},{"issue":"3","key":"5_CR53","doi-asserted-by":"publisher","first-page":"1199","DOI":"10.1109\/TR.2018.2834476","volume":"67","author":"H Liang","year":"2018","unstructured":"Liang, H., Pei, X., Jia, X., Shen, W., Zhang, J.: Fuzzing: state of the art. IEEE Trans. Reliab. 67(3), 1199\u20131218 (2018). https:\/\/doi.org\/10.1109\/TR.2018.2834476","journal-title":"IEEE Trans. Reliab."},{"key":"5_CR54","doi-asserted-by":"crossref","unstructured":"Zhu, X., Wen, S., Camtepe, S., Xiang, Y.: Fuzzing: A survey for roadmap. ACM Comput. Surv. 54(11s), (2022). https:\/\/doi.org\/10.1145\/3512345","DOI":"10.1145\/3512345"},{"issue":"11","key":"5_CR55","doi-asserted-by":"publisher","first-page":"2312","DOI":"10.1109\/TSE.2019.2946563","volume":"47","author":"VJM Man\u00e8s","year":"2021","unstructured":"Man\u00e8s, V.J.M., Han, H., Han, C., Cha, S.K., Egele, M., Schwartz, E.J., Woo, M.: The art, science, and engineering of fuzzing: A survey. IEEE Trans. Software Eng. 47(11), 2312\u20132331 (2021). https:\/\/doi.org\/10.1109\/TSE.2019.2946563","journal-title":"IEEE Trans. Software Eng."},{"issue":"1","key":"5_CR56","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s42400-018-0002-y","volume":"1","author":"J Li","year":"2018","unstructured":"Li, J., Zhao, B., Zhang, C.: Fuzzing: a survey. Cybersecurity 1(1), 1\u201313 (2018)","journal-title":"Cybersecurity"},{"issue":"2","key":"5_CR57","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1145\/3363824","volume":"63","author":"P Godefroid","year":"2020","unstructured":"Godefroid, P.: Fuzzing: Hack, art, and science. Commun. ACM 63(2), 70\u201376 (2020)","journal-title":"Commun. ACM"},{"issue":"8","key":"5_CR58","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pone.0237749","volume":"15","author":"Y Wang","year":"2020","unstructured":"Wang, Y., Jia, P., Liu, L., Huang, C., Liu, Z.: A systematic review of fuzzing based on machine learning techniques. PLOS ONE 15(8), 1\u201337 (2020). https:\/\/doi.org\/10.1371\/journal.pone.0237749","journal-title":"PLOS ONE"},{"issue":"3","key":"5_CR59","doi-asserted-by":"publisher","first-page":"165","DOI":"10.1002\/(SICI)1099-1689(199709)7:3<165::AID-STVR143>3.0.CO;2-U","volume":"7","author":"AJ Offutt","year":"1997","unstructured":"Offutt, A.J., Pan, J.: Automatically detecting equivalent mutants and infeasible paths. Software Test. Verif. Reliab. 7(3), 165\u2013192 (1997). https:\/\/doi.org\/10.1002\/(SICI)1099-1689(199709)7:3%3C165::AID-STVR143%3E3.0.CO;2-U","journal-title":"Software Test. Verif. Reliab."},{"key":"5_CR60","doi-asserted-by":"publisher","unstructured":"Just, R., Kapfhammer, G.M., Schweiggert, F.: Do Redundant Mutants Affect the Effectiveness and Efficiency of Mutation Analysis? In: 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation, pp. 720\u2013725 (2012). https:\/\/doi.org\/10.1109\/ICST.2012.162","DOI":"10.1109\/ICST.2012.162"},{"issue":"1","key":"5_CR61","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1109\/TSE.2013.44","volume":"40","author":"L Madeyski","year":"2014","unstructured":"Madeyski, L., Orzeszyna, W., Torkar, R., J\u00f3zala, M.: Overcoming the equivalent mutant problem: a systematic literature review and a comparative experiment of second order mutation. IEEE Trans. Software Eng. 40(1), 23\u201342 (2014). https:\/\/doi.org\/10.1109\/TSE.2013.44","journal-title":"IEEE Trans. Software Eng."},{"key":"5_CR62","doi-asserted-by":"publisher","unstructured":"Just, R., Kapfhammer, G.M., Schweiggert, F.: Using Non-redundant Mutation Operators and Test Suite Prioritization to Achieve Efficient and Scalable Mutation Analysis. In: 2012 IEEE 23rd International Symposium on Software Reliability Engineering, pp. 11\u201320 (2012). https:\/\/doi.org\/10.1109\/ISSRE.2012.31","DOI":"10.1109\/ISSRE.2012.31"},{"issue":"5-7","key":"5_CR63","doi-asserted-by":"publisher","first-page":"490","DOI":"10.1002\/stvr.1561","volume":"25","author":"R Just","year":"2015","unstructured":"Just, R., Schweiggert, F.: Higher accuracy and lower run time: efficient mutation analysis using non-redundant mutation operators. Software Test. Verif. Reliab. 25(5-7), 490\u2013507 (2015). https:\/\/doi.org\/10.1002\/stvr.1561","journal-title":"Software Test. Verif. Reliab."},{"issue":"2","key":"5_CR64","doi-asserted-by":"publisher","first-page":"182","DOI":"10.1109\/4235.996017","volume":"6","author":"K Deb","year":"2002","unstructured":"Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE Trans. Evol. Comput. 6(2), 182\u2013197 (2002). https:\/\/doi.org\/10.1109\/4235.996017","journal-title":"IEEE Trans. Evol. Comput."},{"issue":"5","key":"5_CR65","doi-asserted-by":"publisher","first-page":"507","DOI":"10.1109\/TSE.2014.2372785","volume":"41","author":"ET Barr","year":"2015","unstructured":"Barr, E.T., Harman, M., McMinn, P., Shahbaz, M., Yoo, S.: The Oracle problem in software testing: a survey. IEEE Trans. Software Eng. 41(5), 507\u2013525 (2015). https:\/\/doi.org\/10.1109\/TSE.2014.2372785","journal-title":"IEEE Trans. Software Eng."},{"key":"5_CR66","doi-asserted-by":"publisher","first-page":"292","DOI":"10.1007\/978-3-030-58793-2_24","volume-title":"Quality of Information and Communications Technology. Communications in Computer and Information Science","author":"M-A Sicilia","year":"2020","unstructured":"Sicilia, M.-A., S\u00e1nchez-Alonso, S., Mora-Cantallops, M., Garc\u00eda-Barriocanal, E.: On the Source Code Structure of Quantum Code: Insights from Q# and QDK. In: Shepperd, M., Abreu, F., Silva, A., P\u00e9rez-Castillo, R. (eds.) Quality of Information and Communications Technology. Communications in Computer and Information Science, pp. 292\u2013299. Springer, Cham (2020). https:\/\/doi.org\/10.1007\/978-3-030-58793-2_24"},{"key":"5_CR67","doi-asserted-by":"publisher","unstructured":"Yamashita, K., Huang, C., Nagappan, M., Kamei, Y., Mockus, A., Hassan, A.E., Ubayashi, N.: Thresholds for Size and Complexity Metrics: A Case Study from the Perspective of Defect Density. In: 2016 IEEE International Conference on Software Quality, Reliability and Security (QRS), pp. 191\u2013201 (2016). https:\/\/doi.org\/10.1109\/QRS.2016.31","DOI":"10.1109\/QRS.2016.31"},{"key":"5_CR68","doi-asserted-by":"crossref","unstructured":"Kumar, A.: Formalization of structural test cases coverage criteria for quantum software testing. Int. J. Theor. Phys. 62, (2023). https:\/\/doi.org\/10.1007\/s10773-022-05271-y","DOI":"10.1007\/s10773-022-05271-y"},{"key":"5_CR69","doi-asserted-by":"crossref","unstructured":"Just, R., Jalali, D., Inozemtseva, L., Ernst, M.D., Holmes, R., Fraser, G.: Are Mutants a Valid Substitute for Real Faults in Software Testing? In: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 654\u2013665. ACM, Hong Kong China (2014). https:\/\/doi.org\/10.1145\/2635868.2635929. https:\/\/dl.acm.org\/doi\/10.1145\/2635868.2635929","DOI":"10.1145\/2635868.2635929"},{"key":"5_CR70","doi-asserted-by":"crossref","unstructured":"Just, R., Jalali, D., Ernst, M.D.: Defects4j: A Database of Existing Faults to Enable Controlled Testing Studies for Java Programs. In: Proceedings of the 2014 International Symposium on Software Testing and Analysis, pp. 437\u2013440. ACM, San Jose, CA, USA (2014). https:\/\/doi.org\/10.1145\/2610384.2628055. https:\/\/dl.acm.org\/doi\/10.1145\/2610384.2628055","DOI":"10.1145\/2610384.2628055"},{"key":"5_CR71","doi-asserted-by":"publisher","unstructured":"Gyimesi, P., Vancsics, B., Stocco, A., Mazinanian, D., Besz\u00e9des, A., Ferenc, R., Mesbah, A.: BugsJS: A Benchmark of JavaScript Bugs. In: 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST), pp. 90\u2013101 (2019). https:\/\/doi.org\/10.1109\/ICST.2019.00019","DOI":"10.1109\/ICST.2019.00019"},{"key":"5_CR72","doi-asserted-by":"crossref","unstructured":"Widyasari, R., Sim, S.Q., Lok, C., Qi, H., Phan, J., Tay, Q., Tan, C., Wee, F., Tan, J.E., Yieh, Y., Goh, B., Thung, F., Kang, H.J., Hoang, T., Lo, D., Ouh, E.L.: BugsInPy: A Database of Existing Bugs in Python Programs to Enable Controlled Testing and Debugging Studies. In: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ESEC\/FSE 2020, pp. 1556\u20131560. Association for Computing Machinery, New York, NY, USA (2020). https:\/\/doi.org\/10.1145\/3368089.3417943","DOI":"10.1145\/3368089.3417943"},{"key":"5_CR73","doi-asserted-by":"publisher","unstructured":"Shamshiri, S., Just, R., Rojas, J.M., Fraser, G., McMinn, P., Arcuri, A.: Do Automatically Generated Unit Tests Find Real Faults? An Empirical Study of Effectiveness and Challenges. In: 2015 30th IEEE\/ACM International Conference on Automated Software Engineering (ASE), pp. 201\u2013211 (2015). https:\/\/doi.org\/10.1109\/ASE.2015.86","DOI":"10.1109\/ASE.2015.86"},{"issue":"2","key":"5_CR74","doi-asserted-by":"publisher","first-page":"36","DOI":"10.1007\/s10664-022-10248-w","volume":"28","author":"S Lukasczyk","year":"2023","unstructured":"Lukasczyk, S., Kroi\u00df, F., Fraser, G.: An empirical study of automated unit test generation for Python. Empirical Software Eng. 28(2), 36 (2023)","journal-title":"Empirical Software Eng."},{"key":"5_CR75","doi-asserted-by":"publisher","unstructured":"Paterson, D., Campos, J., Abreu, R., Kapfhammer, G.M., Fraser, G., McMinn, P.: An Empirical Study on the Use of Defect Prediction for Test Case Prioritization. In: 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST), pp. 346\u2013357 (2019). https:\/\/doi.org\/10.1109\/ICST.2019.00041","DOI":"10.1109\/ICST.2019.00041"},{"key":"5_CR76","doi-asserted-by":"crossref","unstructured":"Miranda, B., Cruciani, E., Verdecchia, R., Bertolino, A.: FAST Approaches to Scalable Similarity-Based Test Case Prioritization. In: Proceedings of the 40th International Conference on Software Engineering. ICSE \u201918, pp. 222\u2013232. Association for Computing Machinery, New York, NY, USA (2018). https:\/\/doi.org\/10.1145\/3180155.3180210","DOI":"10.1145\/3180155.3180210"},{"key":"5_CR77","doi-asserted-by":"publisher","unstructured":"Pearson, S., Campos, J., Just, R., Fraser, G., Abreu, R., Ernst, M.D., Pang, D., Keller, B.: Evaluating and Improving Fault Localization. In: 2017 IEEE\/ACM 39th International Conference on Software Engineering (ICSE), pp. 609\u2013620 (2017). https:\/\/doi.org\/10.1109\/ICSE.2017.62","DOI":"10.1109\/ICSE.2017.62"},{"key":"5_CR78","doi-asserted-by":"crossref","unstructured":"Li, X., Li, W., Zhang, Y., Zhang, L.: DeepFL: Integrating Multiple Fault Diagnosis Dimensions for Deep Fault Localization. In: Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis. ISSTA 2019, pp. 169\u2013180. Association for Computing Machinery, New York, NY, USA (2019). https:\/\/doi.org\/10.1145\/3293882.3330574","DOI":"10.1145\/3293882.3330574"},{"issue":"2","key":"5_CR79","doi-asserted-by":"publisher","first-page":"332","DOI":"10.1109\/TSE.2019.2892102","volume":"47","author":"D Zou","year":"2021","unstructured":"Zou, D., Liang, J., Xiong, Y., Ernst, M.D., Zhang, L.: An empirical study of fault localization families and their combinations. IEEE Trans. Software Eng. 47(2), 332\u2013347 (2021). https:\/\/doi.org\/10.1109\/TSE.2019.2892102","journal-title":"IEEE Trans. Software Eng."},{"key":"5_CR80","doi-asserted-by":"publisher","first-page":"10618","DOI":"10.1109\/ACCESS.2022.3144079","volume":"10","author":"QI Sarhan","year":"2022","unstructured":"Sarhan, Q.I., Besz\u00e9des, A.: A survey of challenges in spectrum-based software fault localization. IEEE Access 10, 10618\u201310639 (2022). https:\/\/doi.org\/10.1109\/ACCESS.2022.3144079","journal-title":"IEEE Access"},{"issue":"6","key":"5_CR81","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1007\/s10664-022-10189-4","volume":"27","author":"R Widyasari","year":"2022","unstructured":"Widyasari, R., Prana, G.A.A., Haryono, S.A., Wang, S., Lo, D.: Real world projects, real faults: evaluating spectrum based fault localization techniques on Python projects. Empirical Software Eng. 27(6), 147 (2022)","journal-title":"Empirical Software Eng."},{"key":"5_CR82","doi-asserted-by":"crossref","unstructured":"Durieux, T., Madeiral, F., Martinez, M., Abreu, R.: Empirical Review of Java Program Repair Tools: A Large-Scale Experiment on 2,141 Bugs and 23,551 Repair Attempts. In: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ESEC\/FSE 2019, pp. 302\u2013313. Association for Computing Machinery, New York, NY, USA (2019). https:\/\/doi.org\/10.1145\/3338906.3338911","DOI":"10.1145\/3338906.3338911"},{"key":"5_CR83","doi-asserted-by":"crossref","unstructured":"Campos, J., Souto, A.: QBugs: A Collection of Reproducible Bugs in Quantum Algorithms and a Supporting Infrastructure to Enable Controlled Quantum Software Testing and Debugging Experiments (2021)","DOI":"10.1109\/Q-SE52541.2021.00013"},{"key":"5_CR84","doi-asserted-by":"crossref","unstructured":"Paltenghi, M., Pradel, M.: Bugs in quantum computing platforms: an empirical study. Proc. ACM Program. Lang. 6(OOPSLA1), (2022). https:\/\/doi.org\/10.1145\/3527330","DOI":"10.1145\/3527330"},{"key":"5_CR85","doi-asserted-by":"crossref","unstructured":"Lewis, C., Lin, Z., Sadowski, C., Zhu, X., Ou, R., Whitehead Jr., E.J.: Does Bug Prediction Support Human Developers? Findings from a Google Case Study. In: Proceedings of the 2013 International Conference on Software Engineering. ICSE \u201913, pp. 372\u2013381. IEEE Press, San Francisco, CA, USA (2013)","DOI":"10.1109\/ICSE.2013.6606583"},{"key":"5_CR86","unstructured":"Freitas, P.A.F.: Software repository mining analytics to estimate software component reliability (2015)"},{"issue":"4\u20135","key":"5_CR87","doi-asserted-by":"publisher","first-page":"531","DOI":"10.1007\/s10664-011-9173-9","volume":"17","author":"M D\u2019Ambros","year":"2012","unstructured":"D\u2019Ambros, M., Lanza, M., Robbes, R.: Evaluating defect prediction approaches: a benchmark and an extensive comparison. Empirical Software Eng. 17(4\u20135), 531\u2013577 (2012). https:\/\/doi.org\/10.1007\/s10664-011-9173-9","journal-title":"Empirical Software Eng."},{"issue":"4","key":"5_CR88","doi-asserted-by":"publisher","first-page":"7346","DOI":"10.1016\/j.eswa.2008.10.027","volume":"36","author":"C Catal","year":"2009","unstructured":"Catal, C., Diri, B.: A systematic review of software fault prediction studies. Expert Syst. Appl. 36(4), 7346\u20137354 (2009). https:\/\/doi.org\/10.1016\/j.eswa.2008.10.027","journal-title":"Expert Syst. Appl."},{"issue":"4","key":"5_CR89","doi-asserted-by":"publisher","first-page":"340","DOI":"10.1109\/TSE.2005.49","volume":"31","author":"TJ Ostrand","year":"2005","unstructured":"Ostrand, T.J., Weyuker, E.J., Bell, R.M.: Predicting the location and number of faults in large software systems. IEEE Trans. Software Eng. 31(4), 340\u2013355 (2005). https:\/\/doi.org\/10.1109\/TSE.2005.49","journal-title":"IEEE Trans. Software Eng."},{"key":"5_CR90","doi-asserted-by":"publisher","unstructured":"Arcuri, A., Campos, J., Fraser, G.: Unit Test Generation During Software Development: EvoSuite Plugins for Maven, IntelliJ and Jenkins. In: 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST), pp. 401\u2013408 (2016). https:\/\/doi.org\/10.1109\/ICST.2016.44","DOI":"10.1109\/ICST.2016.44"},{"key":"5_CR91","doi-asserted-by":"crossref","unstructured":"Campos, J., Riboira, A., Perez, A., Abreu, R.: Gzoltar: An Eclipse Plug-in for Testing and Debugging. In: Proceedings of the 27th IEEE\/ACM International Conference on Automated Software Engineering. ASE \u201912, pp. 378\u2013381. Association for Computing Machinery, New York, NY, USA (2012). https:\/\/doi.org\/10.1145\/2351676.2351752","DOI":"10.1145\/2351676.2351752"}],"container-title":["Quantum Software"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/978-3-031-64136-7_5","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,26]],"date-time":"2024-09-26T13:06:26Z","timestamp":1727355986000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/978-3-031-64136-7_5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024]]},"ISBN":["9783031641350","9783031641367"],"references-count":91,"URL":"https:\/\/doi.org\/10.1007\/978-3-031-64136-7_5","relation":{},"subject":[],"published":{"date-parts":[[2024]]},"assertion":[{"value":"23 August 2024","order":1,"name":"first_online","label":"First Online","group":{"name":"ChapterHistory","label":"Chapter History"}}]}}