{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,3]],"date-time":"2026-03-03T14:38:43Z","timestamp":1772548723057,"version":"3.50.1"},"reference-count":25,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2026,3,3]],"date-time":"2026-03-03T00:00:00Z","timestamp":1772496000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62473201"],"award-info":[{"award-number":["62473201"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Basic Research Program of Jiangsu","award":["BK20231142"],"award-info":[{"award-number":["BK20231142"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Quantum reinforcement learning (QRL) is often evaluated under idealized, noiseless assumptions, yet realistic quantum devices inevitably introduce noise that can severely degrade performance. This paper improves the robustness of quantum deep Q-learning (QDQN) by redesigning the variational quantum circuit (VQC) used in its value-function approximator. Motivated by recent advances in quantum convolutional neural networks (QCNNs), we construct four QCNN-inspired VQC variants (Models A\u2013D) by combining representative QCNN two-qubit building blocks with an explicit fully connected (all-to-all) layer. Using a 10-fold evaluation protocol at a fixed noise level p = 0.005, Model D achieves the best robustness, reducing the mean number of episodes required to reach a target reward from 1981 (baseline) to 1243. Under a stricter success criterion, Model D also doubles the empirically observed noise-tolerance boundary from 0.002 to 0.004. These results indicate that carefully chosen QCNN-style circuit components and connectivity can significantly improve the noise robustness of QDQN-like QRL agents.<\/jats:p>","DOI":"10.3390\/info17030250","type":"journal-article","created":{"date-parts":[[2026,3,3]],"date-time":"2026-03-03T09:45:12Z","timestamp":1772531112000},"page":"250","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["QCNN-Inspired Variational Circuits for Enhanced Noise Robustness in Quantum Deep Q-Learning"],"prefix":"10.3390","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-6366-7510","authenticated-orcid":false,"given":"Louyang","family":"Yu","sequence":"first","affiliation":[{"name":"School of Computer Science, Nanjing University of Information Science and Technology, Nanjing 210044, China"},{"name":"Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology (CICAEET), Nanjing University of Information Science and Technology, Nanjing 210044, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4786-4036","authenticated-orcid":false,"given":"Wenbin","family":"Yu","sequence":"additional","affiliation":[{"name":"School of Computer Science, Nanjing University of Information Science and Technology, Nanjing 210044, China"},{"name":"Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology (CICAEET), Nanjing University of Information Science and Technology, Nanjing 210044, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4448-2617","authenticated-orcid":false,"given":"Yadang","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Computer Science, Nanjing University of Information Science and Technology, Nanjing 210044, China"},{"name":"Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology (CICAEET), Nanjing University of Information Science and Technology, Nanjing 210044, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chengjun","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Computer Science, Nanjing University of Information Science and Technology, Nanjing 210044, China"},{"name":"Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology (CICAEET), Nanjing University of Information Science and Technology, Nanjing 210044, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2026,3,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Sutton, R.S., and Barto, A.G. (1992). Reinforcement Learning, Springer.","DOI":"10.1007\/978-1-4615-3618-5"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1207","DOI":"10.1109\/TSMCB.2008.925743","article-title":"Quantum reinforcement learning","volume":"38","author":"Dong","year":"2008","journal-title":"IEEE Trans. Syst. Man Cybern. Part B"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"130501","DOI":"10.1103\/PhysRevLett.117.130501","article-title":"Quantum-enhanced machine learning","volume":"117","author":"Dunjko","year":"2016","journal-title":"Phys. Rev. Lett."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"505","DOI":"10.1038\/s41586-019-1666-5","article-title":"Quantum supremacy using a programmable superconducting processor","volume":"574","author":"Arute","year":"2019","journal-title":"Nature"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"500","DOI":"10.1038\/s41586-023-06096-3","article-title":"Evidence for the utility of quantum computing before fault tolerance","volume":"618","author":"Kim","year":"2023","journal-title":"Nature"},{"key":"ref_6","first-page":"28362","article-title":"Parametrized quantum policies for reinforcement learning","volume":"34","author":"Jerbi","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"720","DOI":"10.22331\/q-2022-05-24-720","article-title":"Quantum agents in the gym: A variational quantum algorithm for deep q-learning","volume":"6","author":"Skolik","year":"2022","journal-title":"Quantum"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"141007","DOI":"10.1109\/ACCESS.2020.3010470","article-title":"Variational quantum circuits for deep reinforcement learning","volume":"8","author":"Chen","year":"2020","journal-title":"IEEE Access"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"300","DOI":"10.1007\/s11128-023-04051-9","article-title":"An efficient and scalable variational quantum circuits approach for deep reinforcement learning","volume":"22","author":"Bar","year":"2023","journal-title":"Quantum Inf. Process."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1900070","DOI":"10.1002\/qute.201900070","article-title":"Expressibility and entangling capability of parameterized quantum circuits for hybrid quantum-classical algorithms","volume":"2","author":"Sim","year":"2019","journal-title":"Adv. Quantum Technol."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Takahashi, Y., Takeuchi, Y., and Tani, S. (2020). Classically simulating quantum circuits with local depolarizing noise. arXiv.","DOI":"10.1016\/j.tcs.2021.07.025"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2250060","DOI":"10.1142\/S0219477522500602","article-title":"Modeling and simulation of a quantum thermal noise on the qubit","volume":"21","year":"2022","journal-title":"Fluct. Noise Lett."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1273","DOI":"10.1038\/s41567-019-0648-8","article-title":"Quantum convolutional neural networks","volume":"15","author":"Cong","year":"2019","journal-title":"Nat. Phys."},{"key":"ref_14","first-page":"041011","article-title":"Absence of barren plateaus in quantum convolutional neural networks","volume":"11","author":"Pesah","year":"2021","journal-title":"Phys. Rev. X"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1007\/s42484-020-00012-y","article-title":"Quanvolutional neural networks: Powering image recognition with quantum circuits","volume":"2","author":"Henderson","year":"2020","journal-title":"Quantum Mach. Intell."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"044003","DOI":"10.1088\/2058-9565\/ab9f93","article-title":"A quantum deep convolutional neural network for image recognition","volume":"5","author":"Li","year":"2020","journal-title":"Quantum Sci. Technol."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Wei, S., Chen, Y., Zhou, Z., and Long, G. (2021). A Quantum Convolutional Neutral Network on NISQ Devices. arXiv.","DOI":"10.1007\/s43673-021-00030-3"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/s42484-021-00061-x","article-title":"Quantum convolutional neural network for classical data classification","volume":"4","author":"Hur","year":"2022","journal-title":"Quantum Mach. Intell."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Jiang, C., Jiang, C., Chen, D., and Hu, F. (2022). Densely connected neural networks for nonlinear regression. Entropy, 24.","DOI":"10.3390\/e24070876"},{"key":"ref_20","unstructured":"Brockman, G., Cheung, V., Pettersson, L., Schneider, J., Schulman, J., Tang, J., and Zaremba, W. (2016). Openai gym. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1586","DOI":"10.1109\/TKDE.2019.2912815","article-title":"Reliable accuracy estimates from k-fold cross validation","volume":"32","author":"Wong","year":"2019","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Feynman, R.P. (2018). Simulating physics with computers. Feynman and Computation, CRC Press.","DOI":"10.1201\/9780429500459-11"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"033044","DOI":"10.1088\/1367-2630\/ac5b56","article-title":"Performance analysis of a hybrid agent for quantum-accessible reinforcement learning","volume":"24","author":"Hamann","year":"2022","journal-title":"New J. Phys."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"062443","DOI":"10.1103\/PhysRevA.105.062443","article-title":"Robust optimization for quantum reinforcement learning control using partial observations","volume":"105","author":"Jiang","year":"2022","journal-title":"Phys. Rev. A"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1038\/srep00400","article-title":"Projective simulation for artificial intelligence","volume":"2","author":"Briegel","year":"2012","journal-title":"Sci. Rep."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/17\/3\/250\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,3]],"date-time":"2026-03-03T10:33:03Z","timestamp":1772533983000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/17\/3\/250"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,3]]},"references-count":25,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2026,3]]}},"alternative-id":["info17030250"],"URL":"https:\/\/doi.org\/10.3390\/info17030250","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,3]]}}}