{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,23]],"date-time":"2026-03-23T14:43:05Z","timestamp":1774276985623,"version":"3.50.1"},"reference-count":121,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,2,25]],"date-time":"2025-02-25T00:00:00Z","timestamp":1740441600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,2,25]],"date-time":"2025-02-25T00:00:00Z","timestamp":1740441600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Fraunhofer-Institut f\u00fcr Techno- und Wirtschaftsmathematik ITWM"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Quantum Mach. Intell."],"published-print":{"date-parts":[[2025,6]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Methods of artificial intelligence (AI) and especially machine learning (ML) have been growing ever more complex, and at the same time have more and more impact on people\u2019s lives. This leads to explainable AI (XAI) manifesting itself as an important research field that helps humans to better comprehend ML systems. In parallel, quantum machine learning (QML) is emerging with the ongoing improvement of quantum computing hardware combined with its increasing availability via cloud services. QML enables quantum-enhanced ML in which quantum mechanics is exploited to facilitate ML tasks, typically in the form of quantum-classical hybrid algorithms that combine quantum and classical resources. Quantum gates constitute the building blocks of gate-based quantum hardware and form circuits that can be used for quantum computations. For QML applications, quantum circuits are typically parameterized and their parameters are optimized classically such that a suitably defined objective function is minimized. Inspired by XAI, we raise the question of the explainability of such circuits by quantifying the importance of (groups of) gates for specific goals. To this end, we apply the well-established concept of Shapley values. The resulting attributions can be interpreted as explanations for why a specific circuit works well for a given task, improving the understanding of how to construct parameterized (or variational) quantum circuits, and fostering their human interpretability in general. An experimental evaluation on simulators and two superconducting quantum hardware devices demonstrates the benefits of the proposed framework for classification, generative modeling, transpilation, and optimization. Furthermore, our results shed some light on the role of specific gates in popular QML approaches.<\/jats:p>","DOI":"10.1007\/s42484-025-00254-8","type":"journal-article","created":{"date-parts":[[2025,2,25]],"date-time":"2025-02-25T07:44:47Z","timestamp":1740469487000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Explaining quantum circuits with Shapley values: towards explainable quantum machine learning"],"prefix":"10.1007","volume":"7","author":[{"given":"Raoul","family":"Heese","sequence":"first","affiliation":[]},{"given":"Thore","family":"Gerlach","sequence":"additional","affiliation":[]},{"given":"Sascha","family":"M\u00fccke","sequence":"additional","affiliation":[]},{"given":"Sabine","family":"M\u00fcller","sequence":"additional","affiliation":[]},{"given":"Matthias","family":"Jakobs","sequence":"additional","affiliation":[]},{"given":"Nico","family":"Piatkowski","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,2,25]]},"reference":[{"key":"254_CR1","unstructured":"Aha DW (1991) Tic-Tac-Toe endgame data set. https:\/\/archive.ics.uci.edu\/ml\/datasets\/Tic-Tac-Toe+Endgame"},{"key":"254_CR2","unstructured":"Anis MS, et\u00a0al (2019) Qiskit: an open-source framework for quantum computing"},{"key":"254_CR3","doi-asserted-by":"publisher","unstructured":"Aumann RJJ (1994) Economic applications of the Shapley value. Springer Netherlands, Dordrecht, pp 121\u2013133. https:\/\/doi.org\/10.1007\/978-94-017-1656-7_12","DOI":"10.1007\/978-94-017-1656-7_12"},{"key":"254_CR4","unstructured":"Aumann RJ, Shapley LS (1974) Values of non-atomic games, Princeton University Press. http:\/\/www.jstor.org\/stable\/j.ctt13x149m"},{"key":"254_CR5","doi-asserted-by":"crossref","unstructured":"Baheri B, Guan Q, Chaudhary V, Li A, Savino A, Rech P, Carlo SD, Gizopoulos D (2022) Quantum noise in the flow of time: a temporal study of the noise in quantum computers. In: Savino A, Rech P, Carlo SD, Gizopoulos D (eds) 2022 IEEE 28th international symposium on on-line testing and robust system design (IOLTS), pp 1\u20135","DOI":"10.1109\/IOLTS56730.2022.9897404"},{"key":"254_CR6","first-page":"317","volume":"19","author":"J Banzhaf","year":"1965","unstructured":"Banzhaf J (1965) Weighted voting doesn\u2019t work: a mathematical analysis. Rutgers Law Rev 19:317\u2013343","journal-title":"Rutgers Law Rev"},{"key":"254_CR7","doi-asserted-by":"publisher","first-page":"5097","DOI":"10.1007\/s10115-023-01935-1","volume":"65","author":"R Barbudo","year":"2023","unstructured":"Barbudo R, Ventura S, Romero JR (2023) Eight years of automl: categorisation, review and trends. Knowl Inf Syst 65:5097\u20135149. https:\/\/doi.org\/10.1007\/s10115-023-01935-1","journal-title":"Knowl Inf Syst"},{"key":"254_CR8","doi-asserted-by":"publisher","first-page":"1541","DOI":"10.1137\/S0097539796302452","volume":"26","author":"A Barenco","year":"1997","unstructured":"Barenco A et al (1997) Stabilization of quantum computations by symmetrization. SIAM J Comput 26:1541\u20131557. https:\/\/doi.org\/10.1137\/S0097539796302452","journal-title":"SIAM J Comput"},{"key":"254_CR9","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1038\/nature23474","volume":"549","author":"J Biamonte","year":"2017","unstructured":"Biamonte J et al (2017) Quantum machine learning. Nature 549:195\u2013202. https:\/\/doi.org\/10.1038\/nature23474","journal-title":"Nature"},{"key":"254_CR10","doi-asserted-by":"crossref","unstructured":"Brennen GK (2003) An observable measure of entanglement for pure states of multi-qubit systems. https:\/\/arxiv.org\/abs\/quant-ph\/0305094","DOI":"10.26421\/QIC3.6-5"},{"key":"254_CR11","doi-asserted-by":"publisher","first-page":"167902","DOI":"10.1103\/PhysRevLett.87.167902","volume":"87","author":"H Buhrman","year":"2001","unstructured":"Buhrman H, Cleve R, Watrous J, de Wolf R (2001) Quantum fingerprinting. Phys Rev Lett 87:167902. https:\/\/doi.org\/10.1103\/PhysRevLett.87.167902","journal-title":"Phys Rev Lett"},{"key":"254_CR12","unstructured":"Burge I, Barbeau M, Garcia-Alfaro J (2024) A Shapley value estimation speedup for efficient explainable quantum ai. arXiv:2412.14639"},{"key":"254_CR13","doi-asserted-by":"publisher","first-page":"54","DOI":"10.1038\/s41534-019-0168-5","volume":"5","author":"JJ Burnett","year":"2019","unstructured":"Burnett JJ et al (2019) Decoherence benchmarking of superconducting qubits. NPJ Quantum Inf 5:54. https:\/\/doi.org\/10.1038\/s41534-019-0168-5","journal-title":"NPJ Quantum Inf"},{"key":"254_CR14","doi-asserted-by":"publisher","first-page":"921","DOI":"10.22331\/q-2023-02-09-921","volume":"7","author":"FA Calderon-Vargas","year":"2023","unstructured":"Calderon-Vargas FA, Proctor T, Rudinger K, Sarovar M (2023) Quantum circuit debugging and sensitivity analysis via local inversions. Quantum 7:921. https:\/\/doi.org\/10.22331\/q-2023-02-09-921","journal-title":"Quantum"},{"key":"254_CR15","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41467-022-32550-3","volume":"13","author":"MC Caro","year":"2022","unstructured":"Caro MC et al (2022) Generalization in quantum machine learning from few training data. Nat Commun 13:1\u201311. https:\/\/doi.org\/10.1038\/s41467-022-32550-3","journal-title":"Nat Commun"},{"key":"254_CR16","doi-asserted-by":"publisher","first-page":"625","DOI":"10.1038\/s42254-021-00348-9","volume":"3","author":"M Cerezo","year":"2021","unstructured":"Cerezo M et al (2021) Variational quantum algorithms. Nat Rev Phys 3:625\u2013644. https:\/\/doi.org\/10.1038\/s42254-021-00348-9","journal-title":"Nat Rev Phys"},{"key":"254_CR17","doi-asserted-by":"crossref","unstructured":"Chen SY-C (2024) Differentiable quantum architecture search in asynchronous quantum reinforcement learning. arXiv:2407.18202","DOI":"10.1109\/QCE60285.2024.00178"},{"key":"254_CR18","first-page":"02015","volume":"2004","author":"H Chen","year":"2020","unstructured":"Chen H, Zheng G, Ji Y (2020) Generating hierarchical explanations on text classification via feature interaction detection 2004:02015","journal-title":"Generating hierarchical explanations on text classification via feature interaction detection"},{"key":"254_CR19","doi-asserted-by":"crossref","unstructured":"Ciaramella G, Martinelli F, Mercaldo F, Santone A, IJCNN (2023) Exploring quantum machine learning for explainable malware detection. In: 2023 International Joint Conference on Neural Networks (IJCNN). IEEE, pp 1\u20136","DOI":"10.1109\/IJCNN54540.2023.10191964"},{"key":"254_CR20","unstructured":"Coppersmith D (2002) An approximate Fourier transform useful in quantum factoring. arXiv:0201067"},{"key":"254_CR21","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1007\/BF00994018","volume":"20","author":"C Cortes","year":"1995","unstructured":"Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20:273\u2013297. https:\/\/doi.org\/10.1007\/BF00994018","journal-title":"Mach Learn"},{"key":"254_CR22","doi-asserted-by":"publisher","first-page":"032328","DOI":"10.1103\/PhysRevA.100.032328","volume":"100","author":"AW Cross","year":"2019","unstructured":"Cross AW, Bishop LS, Sheldon S, Nation PD, Gambetta JM (2019) Validating quantum computers using randomized model circuits. Phys Rev A 100:032328. https:\/\/doi.org\/10.1103\/PhysRevA.100.032328","journal-title":"Phys Rev A"},{"key":"254_CR23","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1007\/s11128-024-04391-0","volume":"23","author":"MP Cu\u00e9llar","year":"2024","unstructured":"Cu\u00e9llar MP, Pegalajar MC, Cano C (2024) Automatic evolutionary design of quantum rule-based systems and applications to quantum reinforcement learning. Quantum Inf Process 23:179. https:\/\/doi.org\/10.1007\/s11128-024-04391-0","journal-title":"Quantum Inf Process"},{"key":"254_CR24","unstructured":"Diaz-Ortiz Jr M, Kempinski B, Cornelisse D, Bachrach Y, Kachman T (2023) Using cooperative game theory to prune neural networks. arXiv:2311.10468"},{"key":"254_CR25","doi-asserted-by":"publisher","unstructured":"Ding L, Spector L (2022) Evolutionary quantum architecture search for parametrized quantum circuits. In: GECCO (ed) Proceedings of the genetic and evolutionary computation conference companion, GECCO \u201922, 2190-2195. Association for Computing Machinery, New York, NY, USA. https:\/\/doi.org\/10.1145\/3520304.3534012","DOI":"10.1145\/3520304.3534012"},{"key":"254_CR26","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1038\/s41534-022-00570-y","volume":"8","author":"Y Du","year":"2022","unstructured":"Du Y, Huang T, You S, Hsieh M-H, Tao D (2022) Quantum circuit architecture search for variational quantum algorithms. NPJ Quantum Inf 8:62. https:\/\/doi.org\/10.1038\/s41534-022-00570-y","journal-title":"NPJ Quantum Inf"},{"key":"254_CR27","doi-asserted-by":"publisher","unstructured":"Dunjko V, Taylor JM, Briegel HJ (2016) Quantum-enhanced machine learning. Phys Rev Lett 117. https:\/\/doi.org\/10.1103\/PhysRevLett.117.130501","DOI":"10.1103\/PhysRevLett.117.130501"},{"key":"254_CR28","doi-asserted-by":"publisher","unstructured":"Du Y, Tu Z, Yuan X, Tao D (2022) Efficient measure for the expressivity of variational quantum algorithms. Phys Rev Lett 128. https:\/\/doi.org\/10.1103\/PhysRevLett.128.080506","DOI":"10.1103\/PhysRevLett.128.080506"},{"key":"254_CR29","doi-asserted-by":"publisher","unstructured":"Earnest N, Tornow C, Egger DJ (2021) Pulse-efficient circuit transpilation for quantum applications on cross-resonance-based hardware. Phys Rev Res 3. https:\/\/doi.org\/10.1103\/PhysRevResearch.3.043088","DOI":"10.1103\/PhysRevResearch.3.043088"},{"key":"254_CR30","unstructured":"Farhi E, Goldstone J, Gutmann S (2014) A quantum approximate optimization algorithm. arXiv:1411.4028"},{"key":"254_CR31","doi-asserted-by":"publisher","unstructured":"Fatima SS, Wooldridge M, Jennings NR (2007) A randomized method for the Shapley value for the voting game. In: AAMAS (ed) Proceedings of the 6th international joint conference on autonomous agents and multiagent systems, AAMAS \u201907. Association for Computing Machinery, New York, NY, USA. https:\/\/doi.org\/10.1145\/1329125.1329316","DOI":"10.1145\/1329125.1329316"},{"key":"254_CR32","doi-asserted-by":"publisher","first-page":"045011","DOI":"10.1088\/2058-9565\/ad5907","volume":"9","author":"F Flamini","year":"2024","unstructured":"Flamini F, Krumm M, Fiderer LJ, M\u00fcller T, Briegel HJ (2024) Towards interpretable quantum machine learning via single-photon quantum walks. Quantum Sci Technol 9:045011. https:\/\/doi.org\/10.1088\/2058-9565\/ad5907","journal-title":"Quantum Sci Technol"},{"key":"254_CR33","doi-asserted-by":"publisher","unstructured":"Franken L, et\u00a0al (2022) Quantum circuit evolution on NISQ devices. In: CEC (ed) 2022 IEEE congress on evolutionary computation (CEC). IEEE, pp 1\u20138. https:\/\/doi.org\/10.1109\/CEC55065.2022.9870269","DOI":"10.1109\/CEC55065.2022.9870269"},{"key":"254_CR34","unstructured":"Gambetta J (2022) Expanding the IBM Quantum roadmap to anticipate the future of quantum-centric supercomputing. https:\/\/research.ibm.com\/blog\/ibm-quantum-roadmap-2025"},{"key":"254_CR35","unstructured":"Gentinetta G, Thomsen A, Sutter D, Woerner S (2022) The complexity of quantum support vector machines. arXiv:2203.00031"},{"key":"254_CR36","doi-asserted-by":"publisher","unstructured":"Georgopoulos K, Emary C, Zuliani P (2021) Modeling and simulating the noisy behavior of near-term quantum computers. Phys Rev A 104. https:\/\/doi.org\/10.1103\/PhysRevA.104.062432","DOI":"10.1103\/PhysRevA.104.062432"},{"key":"254_CR37","unstructured":"Gil-Fuster E, et\u00a0al (2024) Opportunities and limitations of explaining quantum machine learning. arXiv:2412.14753"},{"key":"254_CR38","doi-asserted-by":"publisher","unstructured":"Giovagnoli A, Tresp V, Ma Y, Schubert M (2023) QNEAT: natural evolution of variational quantum circuit architecture. In: GECCO (ed) Proceedings of the companion conference on genetic and evolutionary computation, GECCO \u201923 Companion. Association for Computing Machinery, New York, NY, USA, pp 647\u2013650. https:\/\/doi.org\/10.1145\/3583133.3590675","DOI":"10.1145\/3583133.3590675"},{"key":"254_CR39","doi-asserted-by":"crossref","unstructured":"Gujju Y, Matsuo A, Raymond R (2024) Quantum machine learning on near-term quantum devices: current state of supervised and unsupervised techniques for real-world applications. Phys Rev Appl 21:067001. https:\/\/link.aps.org\/doi\/10.1103\/PhysRevApplied.21.067001","DOI":"10.1103\/PhysRevApplied.21.067001"},{"key":"254_CR40","doi-asserted-by":"publisher","unstructured":"Hadfield S et al (2019) From the quantum approximate optimization algorithm to a quantum alternating operator ansatz. Algorithms 12:34. https:\/\/doi.org\/10.3390\/a12020034","DOI":"10.3390\/a12020034"},{"key":"254_CR41","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1007\/s00220-009-0873-6","volume":"291","author":"AW Harrow","year":"2009","unstructured":"Harrow AW, Low RA (2009) Random quantum circuits are approximate 2-designs. Commun Math Phys 291:257\u2013302. https:\/\/doi.org\/10.1007\/s00220-009-0873-6","journal-title":"Commun Math Phys"},{"key":"254_CR42","doi-asserted-by":"publisher","unstructured":"Haug T, Bharti K, Kim M (2021) Capacity and quantum geometry of parametrized quantum circuits. PRX Quantum 2. https:\/\/doi.org\/10.1103\/PRXQuantum.2.040309","DOI":"10.1103\/PRXQuantum.2.040309"},{"key":"254_CR43","doi-asserted-by":"publisher","unstructured":"Havl\u00ed\u010dek V et al (2019) Supervised learning with quantum-enhanced feature spaces. Nature 567:209\u2013212. https:\/\/doi.org\/10.1038\/s41586-019-0980-2","DOI":"10.1038\/s41586-019-0980-2"},{"key":"254_CR44","unstructured":"Heese R (2022) Quantum decision trees (qtree). https:\/\/github.com\/RaoulHeese\/qtree"},{"key":"254_CR45","unstructured":"Heese R (2023) Quantum Shapley value toolbox. https:\/\/github.com\/RaoulHeese\/qshaptools"},{"key":"254_CR46","doi-asserted-by":"publisher","first-page":"676","DOI":"10.22331\/q-2022-03-30-676","volume":"6","author":"R Heese","year":"2022","unstructured":"Heese R, Bickert P, Niederle AE (2022) Representation of binary classification trees with binary features by quantum circuits. Quantum 6:676. https:\/\/doi.org\/10.22331\/q-2022-03-30-676","journal-title":"Quantum"},{"key":"254_CR47","doi-asserted-by":"publisher","unstructured":"Heese R, et\u00a0al (2023) Data for: explainable quantum machine learning. https:\/\/doi.org\/10.5281\/zenodo.7603939","DOI":"10.5281\/zenodo.7603939"},{"key":"254_CR48","doi-asserted-by":"crossref","unstructured":"Heese R, M\u00fccke S, Jakobs M, Gerlach T, Piatkowski N (2023) Shapley values with uncertain value functions. arXiv:2301.08086","DOI":"10.1007\/978-3-031-30047-9_13"},{"key":"254_CR49","doi-asserted-by":"publisher","first-page":"210","DOI":"10.1515\/crll.1909.136.210","volume":"1909","author":"E Hellinger","year":"1909","unstructured":"Hellinger E (1909) Neue Begr\u00fcndung der Theorie quadratischer Formen von unendlichvielen Ver\u00e4nderlichen. Journal f\u00fcr die reine und angewandte Mathematik 1909:210\u2013271. https:\/\/doi.org\/10.1515\/crll.1909.136.210","journal-title":"Journal f\u00fcr die reine und angewandte Mathematik"},{"key":"254_CR50","doi-asserted-by":"publisher","unstructured":"Holmes Z, Sharma K, Cerezo M, Coles PJ (2022) Connecting ansatz expressibility to gradient magnitudes and barren plateaus. PRX Quantum 3. https:\/\/doi.org\/10.1103\/PRXQuantum.3.010313","DOI":"10.1103\/PRXQuantum.3.010313"},{"key":"254_CR51","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41467-021-22539-9","volume":"12","author":"H-Y Huang","year":"2021","unstructured":"Huang H-Y et al (2021) Power of data in quantum machine learning. Nat Commun 12:1\u20139. https:\/\/doi.org\/10.1038\/s41467-021-22539-9","journal-title":"Nat Commun"},{"key":"254_CR52","doi-asserted-by":"crossref","unstructured":"Huang X, Marques-Silva J (2023) The inadequacy of Shapley values for explainability. arXiv:2302.08160","DOI":"10.1016\/j.ijar.2023.109112"},{"key":"254_CR53","unstructured":"IBM (2022a) IBM quantum Germany. https:\/\/de.quantum-computing.ibm.com"},{"key":"254_CR54","unstructured":"IBM (2022b) IBM quantum. https:\/\/quantum-computing.ibm.com"},{"key":"254_CR55","unstructured":"Karczmarz A, Mukherjee A, Sankowski P, Wygocki P (2021) Improved feature importance computations for tree models: Shapley vs. Banzhaf. arXiv:2108.04126"},{"key":"254_CR56","doi-asserted-by":"publisher","unstructured":"Khan FS, Solmeyer N, Balu R, Humble TS (2018) Quantum games: a review of the history, current state, and interpretation. Quantum Inf Process 17. https:\/\/doi.org\/10.1007\/s11128-018-2082-8","DOI":"10.1007\/s11128-018-2082-8"},{"key":"254_CR57","doi-asserted-by":"publisher","first-page":"045002","DOI":"10.1088\/2058-9565\/ac79c9","volume":"7","author":"BT Kiani","year":"2022","unstructured":"Kiani BT, Palma GD, Marvian M, Liu Z-W, Lloyd S (2022) Learning quantum data with the quantum earth mover\u2019s distance. Quantum Sci Technol 7:045002. https:\/\/doi.org\/10.1088\/2058-9565\/ac79c9","journal-title":"Quantum Sci Technol"},{"key":"254_CR58","unstructured":"Klau D, Z\u00f6ller M, Tutschku C (2023) Bringing quantum algorithms to automated machine learning: a systematic review of automl frameworks regarding extensibility for qml algorithms. arXiv:2310.04238"},{"key":"254_CR59","unstructured":"Kumar IE, Venkatasubramanian S, Scheidegger C, Friedler S, III HD, Singh A (2020) Problems with Shapley-value-based explanations as feature importance measures. In: III HD, Singh A (eds) Proceedings of the 37th international conference on machine learning, Vol. 119 of Proceedings of Machine Learning Research. PMLR, pp 5491\u20135500. https:\/\/proceedings.mlr.press\/v119\/kumar20e.html"},{"key":"254_CR60","first-page":"01203","volume":"2208","author":"O Kyriienko","year":"2022","unstructured":"Kyriienko O, Magnusson EB (2022) Unsupervised quantum machine learning for fraud detection 2208:01203","journal-title":"Unsupervised quantum machine learning for fraud detection"},{"key":"#cr-split#-254_CR61.1","doi-asserted-by":"crossref","unstructured":"Leon F (2014) Optimizing neural network topology using Shapley value. In: ICSTCC","DOI":"10.1109\/ICSTCC.2014.6982527"},{"key":"#cr-split#-254_CR61.2","unstructured":"(ed) 2014 18th international conference on system theory, control and computing (ICSTCC), pp 862-867"},{"key":"254_CR62","doi-asserted-by":"crossref","unstructured":"Lin H-Y, Tseng H-H, Chen SY-C, Yoo S, Gurrola J (2024) Quantum gradient class activation map for model interpretability. In: Gurrola J (ed), 2024 IEEE workshop on signal processing systems (SiPS). IEEE, pp 165\u2013170","DOI":"10.1109\/SiPS62058.2024.00037"},{"key":"254_CR63","doi-asserted-by":"crossref","unstructured":"Linardatos P, Papastefanopoulos V, Kotsiantis S (2020) Explainable ai: a review of machine learning interpretability methods. Entropy 23. https:\/\/www.mdpi.com\/1099-4300\/23\/1\/18","DOI":"10.3390\/e23010018"},{"key":"254_CR64","doi-asserted-by":"publisher","first-page":"1013","DOI":"10.1038\/s41567-021-01287-z","volume":"17","author":"Y Liu","year":"2021","unstructured":"Liu Y, Arunachalam S, Temme K (2021) A rigorous and robust quantum speed-up in supervised machine learning. Nat Phys 17:1013\u20131017. https:\/\/doi.org\/10.1038\/s41567-021-01287-z","journal-title":"Nat Phys"},{"key":"254_CR65","unstructured":"Lloyd S, Schuld M, Ijaz A, Izaac J, Killoran N (2020) Quantum embeddings for machine learning. arXiv:2001.03622"},{"key":"254_CR66","doi-asserted-by":"publisher","first-page":"415","DOI":"10.1007\/s10100-009-0100-8","volume":"17","author":"S L\u00f3pez","year":"2009","unstructured":"L\u00f3pez S, Saboya M (2009) On the relationship between Shapley and Owen values. Cent Eur J Oper Res 17:415\u2013423. https:\/\/doi.org\/10.1007\/s10100-009-0100-8","journal-title":"Cent Eur J Oper Res"},{"key":"254_CR67","unstructured":"Lundberg SM, Lee S-I (2017) A unified approach to interpreting model predictions. In: Guyon I, et\u00a0al (eds) Advances in neural information processing systems, vol 30, pp 4765\u20134774. http:\/\/papers.nips.cc\/paper\/7062-a-unified-approach-to-interpreting-model-predictions.pdf"},{"key":"254_CR68","unstructured":"Marcinkevi\u010ds R, Vogt JE (2020) Interpretability and explainability: a machine learning zoo mini-tour. https:\/\/arxiv.org\/abs\/2012.01805"},{"key":"254_CR69","doi-asserted-by":"crossref","unstructured":"Maria\u00a0Schuld FP (2018) Supervised learning with quantum computers 1 edn. Quantum Science and Technology, Springer Cham","DOI":"10.1007\/978-3-319-96424-9_1"},{"key":"254_CR70","doi-asserted-by":"crossref","unstructured":"Mercaldo F, et\u00a0al (2022) Towards explainable quantum machine learning for mobile malware detection and classification. Appl Sci 12. https:\/\/www.mdpi.com\/2076-3417\/12\/23\/12025","DOI":"10.3390\/app122312025"},{"key":"254_CR71","first-page":"03464","volume":"2211","author":"N Meyer","year":"2022","unstructured":"Meyer N et al (2022) A survey on quantum reinforcement learning 2211:03464","journal-title":"A survey on quantum reinforcement learning"},{"key":"254_CR72","doi-asserted-by":"publisher","first-page":"4273","DOI":"10.1063\/1.1497700","volume":"43","author":"DA Meyer","year":"2002","unstructured":"Meyer DA, Wallach NR (2002) Global entanglement in multiparticle systems. J Math Phys 43:4273\u20134278. https:\/\/doi.org\/10.1063\/1.1497700","journal-title":"J Math Phys"},{"key":"254_CR73","doi-asserted-by":"publisher","first-page":"3503","DOI":"10.1007\/s10462-021-10088-y","volume":"55","author":"D Minh","year":"2022","unstructured":"Minh D, Wang HX, Li YF, Nguyen TN (2022) Explainable artificial intelligence: a comprehensive review. Artif Intell Rev 55:3503\u20133568. https:\/\/doi.org\/10.1007\/s10462-021-10088-y","journal-title":"Artif Intell Rev"},{"key":"254_CR74","doi-asserted-by":"publisher","unstructured":"Moll N et al (2018) Quantum optimization using variational algorithms on near-term quantum devices. Quantum Sci Technol 3:030503. https:\/\/doi.org\/10.1088\/2058-9565\/aab822","DOI":"10.1088\/2058-9565\/aab822"},{"key":"254_CR75","unstructured":"Molnar C (2022) Interpretable machine learning 2 edn. https:\/\/christophm.github.io\/interpretable-ml-book"},{"key":"254_CR76","doi-asserted-by":"crossref","unstructured":"Mukhamediev RI, et\u00a0al (2022) Review of artificial intelligence and machine learning technologies: classification, restrictions, opportunities and challenges. Mathematics 10. https:\/\/www.mdpi.com\/2227-7390\/10\/15\/2552","DOI":"10.3390\/math10152552"},{"key":"254_CR77","doi-asserted-by":"crossref","unstructured":"Murali P, Baker JM, Abhari AJ, Chong FT, Martonosi M (2019) Noise-adaptive compiler mappings for noisy intermediate-scale quantum computers. arXiv:1901.11054","DOI":"10.1145\/3297858.3304075"},{"key":"254_CR78","doi-asserted-by":"crossref","unstructured":"Nakaji K, Yamamoto N (2021) Expressibility of the alternating layered ansatz for quantum computation. Quantum 5:434. https:\/\/doi.org\/10.22331\/q-2021-04-19-434","DOI":"10.22331\/q-2021-04-19-434"},{"key":"254_CR79","doi-asserted-by":"publisher","unstructured":"Nation PD, Kang H, Sundaresan N, Gambetta JM (2021) Scalable mitigation of measurement errors on quantum computers. PRX Quantum 2. https:\/\/doi.org\/10.1103\/PRXQuantum.2.040326","DOI":"10.1103\/PRXQuantum.2.040326"},{"key":"254_CR80","unstructured":"Nation P, Treinish M, Possel C (2022) Mapomatic. https:\/\/github.com\/Qiskit-Partners\/mapomatic"},{"key":"254_CR81","volume-title":"Quantum computation and quantum information: 10th anniversary edition","author":"MA Nielsen","year":"2011","unstructured":"Nielsen MA, Chuang IL (2011) Quantum computation and quantum information: 10th anniversary edition, 10th edn. Cambridge University Press, USA","edition":"10"},{"key":"254_CR82","doi-asserted-by":"crossref","unstructured":"Oloulade BM et al (2025) Shapley-guided pruning for efficient graph neural architecture prediction in distributed learning environments. Inf Sci 695:121695. https:\/\/www.sciencedirect.com\/science\/article\/pii\/S0020025524016098","DOI":"10.1016\/j.ins.2024.121695"},{"key":"254_CR83","doi-asserted-by":"crossref","unstructured":"Owen G, Henn R, Moeschlin O (1977) Values of games with a priori unions. In: Henn R, Moeschlin O (eds) Mathematical economics and game theory. Springer Berlin Heidelberg, Berlin, Heidelberg, pp 76\u201388","DOI":"10.1007\/978-3-642-45494-3_7"},{"key":"254_CR84","doi-asserted-by":"publisher","first-page":"52","DOI":"10.1007\/s42484-024-00191-y","volume":"6","author":"L Pira","year":"2024","unstructured":"Pira L, Ferrie C (2024) On the interpretability of quantum neural networks. Quantum Mach Intell 6:52. https:\/\/doi.org\/10.1007\/s42484-024-00191-y","journal-title":"Quantum Mach Intell"},{"key":"254_CR85","unstructured":"Powell MJD (2007) A view of algorithms for optimization without derivatives. Tech. Rep. 5, Institute of Mathematics and its Applications"},{"key":"254_CR86","unstructured":"Power L, Guha K (2024) Feature importance and explainability in quantum machine learning. arXiv:2405.08917"},{"key":"254_CR87","doi-asserted-by":"publisher","unstructured":"Preskill J (2018) Quantum computing in the NISQ era and beyond. Quantum 2:79. https:\/\/doi.org\/10.22331\/q-2018-08-06-79","DOI":"10.22331\/q-2018-08-06-79"},{"key":"254_CR88","doi-asserted-by":"crossref","unstructured":"Rozemberczki B, et\u00a0al (2022) The Shapley value in machine learning. arXiv:2202.05594","DOI":"10.24963\/ijcai.2022\/778"},{"key":"254_CR89","doi-asserted-by":"publisher","first-page":"2862","DOI":"10.1109\/TVCG.2024.3388557","volume":"30","author":"S Ruan","year":"2024","unstructured":"Ruan S et al (2024) Violet: visual analytics for explainable quantum neural networks. IEEE Trans Vis Comput Graph 30:2862\u20132874","journal-title":"IEEE Trans Vis Comput Graph"},{"key":"254_CR90","unstructured":"Rybotycki T, Gawron P (2024) Aqmlator \u2013 an auto quantum machine learning e-platform. arXiv:2409.18338"},{"key":"254_CR91","doi-asserted-by":"crossref","unstructured":"Saeed W, Omlin C (2023) Explainable AI (XAI): a systematic meta-survey of current challenges and future opportunities. Knowl Based Syst 263:110273. https:\/\/www.sciencedirect.com\/science\/article\/pii\/S0950705123000230","DOI":"10.1016\/j.knosys.2023.110273"},{"key":"254_CR92","doi-asserted-by":"crossref","unstructured":"Salehin I et al (2024) AutoML: a systematic review on automated machine learning with neural architecture search. J Inf Intell 2:52\u201381. https:\/\/www.sciencedirect.com\/science\/article\/pii\/S2949715923000604","DOI":"10.1016\/j.jiixd.2023.10.002"},{"key":"254_CR93","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1007\/s10462-024-11058-w","volume":"58","author":"S Salmani Pour Avval","year":"2025","unstructured":"Salmani Pour Avval S, Eskue ND, Groves RM, Yaghoubi V (2025) Systematic review on neural architecture search. Artif Intell Rev 58:73. https:\/\/doi.org\/10.1007\/s10462-024-11058-w","journal-title":"Artif Intell Rev"},{"key":"254_CR94","doi-asserted-by":"publisher","first-page":"321","DOI":"10.22331\/q-2020-09-11-321","volume":"4","author":"M Sarovar","year":"2020","unstructured":"Sarovar M et al (2020) Detecting crosstalk errors in quantum information processors. Quantum 4:321. https:\/\/doi.org\/10.22331\/q-2020-09-11-321","journal-title":"Quantum"},{"key":"254_CR95","first-page":"11020","volume":"2101","author":"M Schuld","year":"2021","unstructured":"Schuld M (2021) Supervised quantum machine learning models are kernel methods 2101:11020","journal-title":"Supervised quantum machine learning models are kernel methods"},{"key":"254_CR96","doi-asserted-by":"crossref","unstructured":"Schuld M, Killoran N (2022) Is quantum advantage the right goal for quantum machine learning?. arXiv:2203.01340","DOI":"10.1103\/PRXQuantum.3.030101"},{"key":"254_CR97","doi-asserted-by":"publisher","unstructured":"Schuld M, Sweke R, Meyer JJ (2021) Effect of data encoding on the expressive power of variational quantum-machine-learning models. Phys Rev A 103. https:\/\/doi.org\/10.1103\/PhysRevA.103.032430","DOI":"10.1103\/PhysRevA.103.032430"},{"key":"254_CR98","first-page":"307","volume":"2","author":"LS Shapley","year":"1953","unstructured":"Shapley LS (1953) A value for n-person games. Contributions to the Theory of Games 2:307\u2013317","journal-title":"Contributions to the Theory of Games"},{"key":"254_CR99","doi-asserted-by":"publisher","unstructured":"Sim S, Johnson PD, Aspuru-Guzik A (2019) Expressibility and entangling capability of parameterized quantum circuits for hybrid quantum-classical algorithms. Ad Quantum Technol 2:1900070. https:\/\/doi.org\/10.1002\/qute.201900070","DOI":"10.1002\/qute.201900070"},{"key":"254_CR100","doi-asserted-by":"publisher","first-page":"025019","DOI":"10.1088\/2058-9565\/abe107","volume":"6","author":"S Sim","year":"2021","unstructured":"Sim S, Romero J, Gonthier JF, Kunitsa AA (2021) Adaptive pruning-based optimization of parameterized quantum circuits. Quantum Sci Technol 6:025019. https:\/\/doi.org\/10.1088\/2058-9565\/abe107","journal-title":"Quantum Sci Technol"},{"key":"254_CR101","unstructured":"Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929\u20131958. http:\/\/jmlr.org\/papers\/v15\/srivastava14a.html"},{"key":"254_CR102","doi-asserted-by":"crossref","unstructured":"Stier J, Gianini G, Granitzer M, Ziegler K (2018) Analysing neural network topologies: a game theoretic approach. Proc Comput Sci 126:234\u2013243. https:\/\/www.sciencedirect.com\/science\/article\/pii\/S187705091831233X. Knowledge-Based and Intelligent Information & Engineering Systems: Proceedings of the 22nd International Conference, KES-2018, Belgrade, Serbia","DOI":"10.1016\/j.procs.2018.07.257"},{"key":"254_CR103","unstructured":"Strumbelj E, Kononenko I (2010) An efficient explanation of individual classifications using game theory. J Mach Learn Res 11:1\u201318. Publisher: JMLR. org"},{"key":"254_CR104","unstructured":"Sundararajan M, Najmi A, III HD, Singh A (2020) The many Shapley values for model explanation. In: III HD, Singh A (eds) International conference on machine learning, Vol. 119 of Proceedings of Machine Learning Research. PMLR, pp 9269\u20139278"},{"key":"254_CR105","unstructured":"Sundararajan M, Taly A, Yan Q, Precup D, Teh YW (2017) Axiomatic attribution for deep networks. in: Precup D, Teh YW (eds) Proceedings of the 34th international conference on machine learning, Vol.\u00a070 of Proceedings of Machine Learning Research. PMLR, pp 3319\u20133328"},{"key":"254_CR106","unstructured":"Sun M, Geller MR (2018) Efficient characterization of correlated SPAM errors. arXiv:1810.10523"},{"key":"254_CR107","unstructured":"Tannu SS, Qureshi MK (2018) A case for variability-aware policies for nisq-era quantum computers. arXiv:1805.10224"},{"key":"254_CR108","doi-asserted-by":"crossref","unstructured":"Tian J, Yang W (2024) Explainable quantum neural networks: example-based and feature-based methods. Electronics 13. https:\/\/www.mdpi.com\/2079-9292\/13\/20\/4136","DOI":"10.3390\/electronics13204136"},{"key":"254_CR109","unstructured":"Tucci RR (2005) An introduction to Cartan\u2019s KAK decomposition for QC programmers. https:\/\/arxiv.org\/abs\/quant-ph\/0507171"},{"key":"254_CR110","doi-asserted-by":"publisher","first-page":"851","DOI":"10.1613\/jair.1.13283","volume":"74","author":"G Van den Broeck","year":"2022","unstructured":"Van den Broeck G, Lykov A, Schleich M, Suciu D (2022) On the tractability of SHAP explanations. J Artif Intell Res 74:851\u2013886","journal-title":"J Artif Intell Res"},{"key":"254_CR111","unstructured":"Wack A, et\u00a0al (2021) Quality, speed, and scale: three key attributes to measure the performance of near-term quantum computers. arXiv:2110.14108"},{"key":"254_CR112","unstructured":"Weigold M, Barzen J, Leymann F, Salm M Wirfs-Brock R (2020) Data encoding patterns for quantum computing. In: Wirfs-Brock R (ed) Proceedings of the 27th conference on pattern languages of programs, PLoP \u201920. The Hillside Group, USA, pp 1\u201311"},{"key":"254_CR113","doi-asserted-by":"crossref","unstructured":"Wilson E, Singh S, Mueller F, M\u00fcller HA, Byrd G, Culhane C, DeBenedictis E, Humble T (2020) Just-in-time quantum circuit transpilation reduces noise. In: M\u00fcller HA, Byrd G, Culhane C, DeBenedictis E, Humble T (eds) 2020 IEEE international conference on quantum computing and engineering (QCE), pp 345\u2013355","DOI":"10.1109\/QCE49297.2020.00050"},{"key":"254_CR114","doi-asserted-by":"crossref","unstructured":"Winter E (2002) The Shapley value. In: Aumann R, Hart S (eds) Vol.\u00a03 of Handbook of game theory with economic applications. Elsevier, pp 2025\u20132054. https:\/\/www.sciencedirect.com\/science\/article\/pii\/S1574000502030163","DOI":"10.1016\/S1574-0005(02)03016-3"},{"key":"254_CR115","doi-asserted-by":"crossref","unstructured":"Xiao H, Wang Z, Zhu Z, Zhou J, Lu J (2022) Shapley-NAS: discovering operation contribution for neural architecture search. arXiv:2206.09811","DOI":"10.1109\/CVPR52688.2022.01159"},{"key":"254_CR116","doi-asserted-by":"publisher","unstructured":"Zambrano L, Pereira L, Niklitschek S, Delgado A (2020) Estimation of pure quantum states in high dimension at the limit of quantum accuracy through complex optimization and statistical inference. Sci Rep 10:12781. https:\/\/doi.org\/10.1038\/s41598-020-69646-z","DOI":"10.1038\/s41598-020-69646-z"},{"key":"254_CR117","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1631\/FITEE.1700808","volume":"19","author":"Q-S Zhang","year":"2018","unstructured":"Zhang Q-S, Zhu S-C (2018) Visual interpretability for deep learning: a survey. Front Inf Technol Electron Eng 19:27\u201339. https:\/\/doi.org\/10.1631\/FITEE.1700808","journal-title":"Front Inf Technol Electron Eng"},{"key":"254_CR118","unstructured":"Zhang V, Nation PD (2023) Characterizing quantum processors using discrete time crystals. arXiv:2301.07625"},{"key":"254_CR119","doi-asserted-by":"publisher","first-page":"041038","DOI":"10.1103\/PhysRevX.10.041038","volume":"10","author":"Y Zhou","year":"2020","unstructured":"Zhou Y, Stoudenmire EM, Waintal X (2020) What limits the simulation of quantum computers? Phys Rev X 10:041038. https:\/\/doi.org\/10.1103\/PhysRevX.10.041038","journal-title":"Phys Rev X"},{"key":"254_CR120","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1038\/s41534-019-0223-2","volume":"5","author":"C Zoufal","year":"2019","unstructured":"Zoufal C, Lucchi A, Woerner S (2019) Quantum generative adversarial networks for learning and loading random distributions. NPJ Quantum Inf 5:103. https:\/\/doi.org\/10.1038\/s41534-019-0223-2","journal-title":"NPJ Quantum Inf"}],"container-title":["Quantum Machine Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s42484-025-00254-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s42484-025-00254-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s42484-025-00254-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,26]],"date-time":"2025-06-26T14:42:04Z","timestamp":1750948924000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s42484-025-00254-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,25]]},"references-count":121,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,6]]}},"alternative-id":["254"],"URL":"https:\/\/doi.org\/10.1007\/s42484-025-00254-8","relation":{},"ISSN":["2524-4906","2524-4914"],"issn-type":[{"value":"2524-4906","type":"print"},{"value":"2524-4914","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,25]]},"assertion":[{"value":"12 August 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 February 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 February 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"27"}}