{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T16:21:23Z","timestamp":1784218883184,"version":"3.55.0"},"reference-count":56,"publisher":"Wiley","issue":"9","license":[{"start":{"date-parts":[[2024,8,4]],"date-time":"2024-08-04T00:00:00Z","timestamp":1722729600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["advanced.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Advanced Intelligent Systems"],"published-print":{"date-parts":[[2024,9]]},"abstract":"<jats:p>As the third\u2010generation network succeeding artificial neural networks (ANNs), spiking neural networks (SNNs) offer high robustness and low energy consumption. Inspired by biological systems, the limitations of low robustness and high\u2010power consumption in deep reinforcement learning (DRL) are addressed by introducing SNNs. The Brain Q\u2010network (BrainQN) is proposed, which replaces the neurons in the classic Deep Q\u2010learning (DQN) algorithm with SNN neurons. BrainQN is trained using surrogate gradient learning (SGL) and ANN\u2010to\u2010SNN conversion methods. Robustness tests with input noise reveal BrainQN's superior performance, achieving an 82.14% increase in rewards under low noise and 71.74% under high noise compared to DQN. These findings highlight BrainQN's robustness and superior performance in noisy environments, supporting its application in complex scenarios. SGL\u2010trained BrainQN is more robust than ANN\u2010to\u2010SNN conversion under high noise. The differences in network output correlations between noisy and original inputs, along with training algorithm distinctions, explain this phenomenon. BrainQN successfully transitioned from a simulated Pong environment to a ball\u2010catching robot with dynamic vision sensors (DVS). On the neuromorphic chip PAICORE, it shows significant advantages in latency and power consumption compared to Jetson Xavier NX.<\/jats:p>","DOI":"10.1002\/aisy.202400075","type":"journal-article","created":{"date-parts":[[2024,8,5]],"date-time":"2024-08-05T00:38:46Z","timestamp":1722818326000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["BrainQN: Enhancing the Robustness of Deep Reinforcement Learning with Spiking Neural Networks"],"prefix":"10.1002","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0929-0687","authenticated-orcid":false,"given":"Shuo","family":"Feng","sequence":"first","affiliation":[{"name":"School of Software &amp; Microelectronics Peking University  Beijing 102600 China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4724-7065","authenticated-orcid":false,"given":"Jian","family":"Cao","sequence":"additional","affiliation":[{"name":"School of Software &amp; Microelectronics Peking University  Beijing 102600 China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zehong","family":"Ou","sequence":"additional","affiliation":[{"name":"School of Software &amp; Microelectronics Peking University  Beijing 102600 China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guang","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Software &amp; Microelectronics Peking University  Beijing 102600 China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yi","family":"Zhong","sequence":"additional","affiliation":[{"name":"Key Laboratory of Microelectronic Devices and Circuits (MoE) MPW Center School of Integrated Circuits Peking University  Beijing 100871 China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zilin","family":"Wang","sequence":"additional","affiliation":[{"name":"Key Laboratory of Microelectronic Devices and Circuits (MoE) MPW Center School of Integrated Circuits Peking University  Beijing 100871 China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Juntong","family":"Yan","sequence":"additional","affiliation":[{"name":"School of Software &amp; Microelectronics Peking University  Beijing 102600 China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jue","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Software &amp; Microelectronics Peking University  Beijing 102600 China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bingsen","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Software &amp; Microelectronics Peking University  Beijing 102600 China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chenglong","family":"Zou","sequence":"additional","affiliation":[{"name":"Peking University Chongqing Research Institute of Big Data  Chongqing 400030 China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zebang","family":"Feng","sequence":"additional","affiliation":[{"name":"School of Software &amp; Microelectronics Peking University  Beijing 102600 China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4951-4286","authenticated-orcid":false,"given":"Yuan","family":"Wang","sequence":"additional","affiliation":[{"name":"Key Laboratory of Microelectronic Devices and Circuits (MoE) MPW Center School of Integrated Circuits Peking University  Beijing 100871 China"},{"name":"Beijing Advanced Innovation Center for Integrated Circuits  Beijing 100871 China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2024,8,4]]},"reference":[{"key":"e_1_2_10_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2949662"},{"key":"e_1_2_10_3_1","unstructured":"V.Mnih K.Kavukcuoglu D.Silver A.Graves I.Antonoglou D.Wierstra M.Riedmiller \u00a0 (Preprint) arXiv:1312.5602 v1 Submitted: Dec2013."},{"key":"e_1_2_10_4_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature14236"},{"key":"e_1_2_10_5_1","doi-asserted-by":"crossref","unstructured":"S. S.Mousavi M.Schukat E.Howley inProc. of SAI Intelligent Systems Conf. (IntelliSys) 2016Springer International Publishing Londonvol.22018 pp.426\u2013440.","DOI":"10.1007\/978-3-319-56991-8_32"},{"key":"e_1_2_10_6_1","doi-asserted-by":"publisher","DOI":"10.1561\/2200000071"},{"key":"e_1_2_10_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/COMST.2019.2916583"},{"key":"e_1_2_10_8_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-019-1724-z"},{"key":"e_1_2_10_9_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature16961"},{"key":"e_1_2_10_10_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.aar6404"},{"key":"e_1_2_10_11_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-021-04357-7"},{"key":"e_1_2_10_12_1","doi-asserted-by":"crossref","unstructured":"J.Collins D.Howard J.Leitner in2019 Int. Conf. on Robotics and Automation (ICRA) IEEE Piscataway NJ 2019 pp.6706\u20136712.","DOI":"10.1109\/ICRA.2019.8793591"},{"key":"e_1_2_10_13_1","unstructured":"M.Wulfmeier I.Posner P.Abbeel inConf. on Robot Learning PMLR California2017 pp.281\u2013290."},{"key":"e_1_2_10_14_1","unstructured":"Z.Jiang S.Luo inInt. Conf. on Machine Learning PMLR Osaka 2019 pp.3110\u20133119."},{"key":"e_1_2_10_15_1","unstructured":"S.Huang N.Papernot I.Goodfellow Y.Duan P.Abbeel \u00a0(Preprint) arXiv:1702.02284 v1 Submitted: Feb2017."},{"key":"e_1_2_10_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCDS.2020.2974509"},{"key":"e_1_2_10_17_1","doi-asserted-by":"publisher","DOI":"10.1002\/ail2.45"},{"key":"e_1_2_10_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2019.02.006"},{"key":"e_1_2_10_19_1","doi-asserted-by":"publisher","DOI":"10.1063\/1.2203409"},{"key":"e_1_2_10_20_1","doi-asserted-by":"publisher","DOI":"10.1038\/nrn1668"},{"key":"e_1_2_10_21_1","doi-asserted-by":"publisher","DOI":"10.1162\/089976602760407955"},{"key":"e_1_2_10_22_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-019-1677-2"},{"key":"e_1_2_10_23_1","doi-asserted-by":"publisher","DOI":"10.1002\/aisy.202200210"},{"key":"e_1_2_10_24_1","doi-asserted-by":"crossref","unstructured":"E.Orhan 2012 3 1.","DOI":"10.1501\/Andl_0000000390"},{"key":"e_1_2_10_25_1","doi-asserted-by":"crossref","unstructured":"S.Sharmin P.Panda S. S.Sarwar C.Lee W.Ponghiran K.Roy in2019 Int. Joint Conf. on Neural Networks (IJCNN) IEEE Piscataway NJ 2019 pp.1\u20138.","DOI":"10.1109\/IJCNN.2019.8851732"},{"key":"e_1_2_10_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2021.3106961"},{"key":"e_1_2_10_27_1","doi-asserted-by":"crossref","unstructured":"S.Sharmin N.Rathi P.Panda K.Roy inComputer Vision\u2013ECCV 2020: 16th European Conf. Glasgow August 23\u201328 2020 Proceedings Part XXIX 16 Springer International Publishing2020 pp.399\u2013414.","DOI":"10.1007\/978-3-030-58526-6_24"},{"key":"e_1_2_10_28_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11071-015-2190-2"},{"key":"e_1_2_10_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/MM.2018.112130359"},{"key":"e_1_2_10_30_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.1254642"},{"key":"e_1_2_10_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2019.2903009"},{"key":"e_1_2_10_32_1","doi-asserted-by":"crossref","unstructured":"A.Amir P.Datta W. P.Risk A. S.Cassidy J. A.Kusnitz S. K.Esser A.Andreopoulos T. M.Wong M.Flickner R.Alvarez\u2010lcaza E.McQuinn B.Shaw N.Pass D. S.Modha inThe 2013 Int. Joint Conf. on Neural Networks (IJCNN) IEEE Piscataway NJ2013 pp.1\u201310.","DOI":"10.1109\/IJCNN.2013.6707078"},{"key":"e_1_2_10_33_1","doi-asserted-by":"crossref","unstructured":"P.Merolla J.Arthur F.Akopyan N.Imam R.Manohar D. S.Modha in2011 IEEE Custom Integrated Circuits Conf. (CICC) IEEE Piscataway NJ 2011 pp.1\u20134.","DOI":"10.1109\/CICC.2011.6055294"},{"key":"e_1_2_10_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCAD.2015.2474396"},{"key":"e_1_2_10_35_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-019-1424-8"},{"key":"e_1_2_10_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSSC.2020.2970709"},{"key":"e_1_2_10_37_1","first-page":"2655","volume":"68","author":"Kuang Y.","year":"2021","journal-title":"IEEE Trans. Circuits Syst. II: Express Briefs"},{"key":"e_1_2_10_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/TBCAS.2017.2759700"},{"key":"e_1_2_10_39_1","doi-asserted-by":"publisher","DOI":"10.1002\/aisy.202300132"},{"key":"e_1_2_10_40_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-019-1924-6"},{"key":"e_1_2_10_41_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2019.09.036"},{"key":"e_1_2_10_42_1","doi-asserted-by":"publisher","DOI":"10.3389\/fncir.2015.00085"},{"key":"e_1_2_10_43_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1003024"},{"key":"e_1_2_10_44_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2019.08.009"},{"key":"e_1_2_10_45_1","doi-asserted-by":"publisher","DOI":"10.4018\/978-1-4666-1592-2.ch001"},{"key":"e_1_2_10_46_1","first-page":"9816","volume":"35","author":"Tan W.","year":"2021","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"e_1_2_10_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2022.3198259"},{"key":"e_1_2_10_48_1","unstructured":"D.Chen P.Peng T.Huang Y.Tian \u00a0(Preprint) arXiv:2201.09754 v1 Submitted: Jan2022."},{"key":"e_1_2_10_49_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2022.108257"},{"key":"e_1_2_10_50_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.275.5297.213"},{"key":"e_1_2_10_51_1","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2018.00331"},{"key":"e_1_2_10_52_1","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2017.00682"},{"key":"e_1_2_10_53_1","unstructured":"Y.Li S.Deng X.Dong R.Gong S.Gu inInt. Conf. on Machine Learning PMLR London2021 pp.6316\u20136325."},{"key":"e_1_2_10_54_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2021.07.091"},{"key":"e_1_2_10_55_1","doi-asserted-by":"crossref","unstructured":"X.Cheng Y.Hao J.Xu B.Xu inIJCAI Yokohama2020 pp.1519\u20131525.","DOI":"10.24963\/ijcai.2020\/211"},{"key":"e_1_2_10_56_1","first-page":"10244","volume":"35","author":"Liang L.","year":"2022","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_2_10_57_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2021.3117040"}],"container-title":["Advanced Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/advanced.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/aisy.202400075","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T00:00:56Z","timestamp":1759881656000},"score":1,"resource":{"primary":{"URL":"https:\/\/advanced.onlinelibrary.wiley.com\/doi\/10.1002\/aisy.202400075"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,4]]},"references-count":56,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2024,9]]}},"alternative-id":["10.1002\/aisy.202400075"],"URL":"https:\/\/doi.org\/10.1002\/aisy.202400075","archive":["Portico"],"relation":{},"ISSN":["2640-4567","2640-4567"],"issn-type":[{"value":"2640-4567","type":"print"},{"value":"2640-4567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,4]]},"assertion":[{"value":"2024-01-28","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-08-04","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"2400075"}}