{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,16]],"date-time":"2026-05-16T20:19:12Z","timestamp":1778962752046,"version":"3.51.4"},"reference-count":34,"publisher":"MIT Press - Journals","issue":"12","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Neural Computation"],"published-print":{"date-parts":[[2019,12]]},"abstract":"<jats:p> Though succeeding in solving various learning tasks, most existing reinforcement learning (RL) models have failed to take into account the complexity of synaptic plasticity in the neural system. Models implementing reinforcement learning with spiking neurons involve only a single plasticity mechanism. Here, we propose a neural realistic reinforcement learning model that coordinates the plasticities of two types of synapses: stochastic and deterministic. The plasticity of the stochastic synapse is achieved by the hedonistic rule through modulating the release probability of synaptic neurotransmitter, while the plasticity of the deterministic synapse is achieved by a variant of a reward-modulated spike-timing-dependent plasticity rule through modulating the synaptic strengths. We evaluate the proposed learning model on two benchmark tasks: learning a logic gate function and the 19-state random walk problem. Experimental results show that the coordination of diverse synaptic plasticities can make the RL model learn in a rapid and stable form. <\/jats:p>","DOI":"10.1162\/neco_a_01238","type":"journal-article","created":{"date-parts":[[2019,10,15]],"date-time":"2019-10-15T21:52:21Z","timestamp":1571176341000},"page":"2368-2389","source":"Crossref","is-referenced-by-count":20,"title":["Reinforcement Learning in Spiking Neural Networks with Stochastic and Deterministic Synapses"],"prefix":"10.1162","volume":"31","author":[{"given":"Mengwen","family":"Yuan","sequence":"first","affiliation":[{"name":"College of Computer Science, Sichuan University, Chengdu 610065, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xi","family":"Wu","sequence":"additional","affiliation":[{"name":"College of Computer Science, Sichuan University, Chengdu 610065, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rui","family":"Yan","sequence":"additional","affiliation":[{"name":"College of Computer Science, Sichuan University, Chengdu 610065, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Huajin","family":"Tang","sequence":"additional","affiliation":[{"name":"College of Computer Science, Sichuan University, Chengdu 610065, China, and College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.18-24-10464.1998"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1002\/9781119170174.epcn513"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.1016\/j.conb.2008.08.003"},{"key":"B4","volume-title":"Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence","author":"De Asis K.","year":"2018"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-15822-3_21"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1152\/jn.00364.2007"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2007.19.6.1468"},{"issue":"172","key":"B8","volume":"9","author":"Fremaux N.","year":"2015","journal-title":"Frontiers in Neural Circuits"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1003024"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.neuro.29.051605.113038"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.1109\/MCI.2016.2532268"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.3874-10.2010"},{"issue":"99","key":"B13","first-page":"241","volume":"10","author":"Neftci E. O.","year":"2016","journal-title":"Front. Neurosci."},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1162\/NECO_a_00387"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.4402-07.2008"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.3389\/fnsyn.2010.00146"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pcbi.1001133"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2008.08-07-593"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1126\/science.275.5306.1593"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1016\/S0896-6273(03)00761-X"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1016\/S0092-8674(05)80028-5"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.1007\/BF00115009"},{"key":"B23","volume-title":"Reinforcement learning: An introduction","author":"Sutton R. S.","year":"2018"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.5962\/bhl.title.55072"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1371\/annotation\/307ea250-3792-4ceb-b905-162d86c96baf"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.3508-05.2005"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1038\/nn.2276"},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.69.041909"},{"key":"B29","author":"Yang L.","year":"2018","journal-title":"A unified approach for multistep temporal-difference learning with eligibility traces in reinforcement learning"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2013.2245677"},{"key":"B31","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2015.2416771"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.1038\/ncomms7922"},{"key":"B33","doi-asserted-by":"publisher","DOI":"10.1162\/neco_a_01086"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0900546106"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/neco_a_01238","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:43:23Z","timestamp":1615585403000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/31\/12\/2368-2389\/95611"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,12]]},"references-count":34,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2019,12]]}},"alternative-id":["10.1162\/neco_a_01238"],"URL":"https:\/\/doi.org\/10.1162\/neco_a_01238","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,12]]}}}