{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,5,13]],"date-time":"2025-05-13T21:56:12Z","timestamp":1747173372970,"version":"3.40.5"},"reference-count":60,"publisher":"Cambridge University Press (CUP)","issue":"9","license":[{"start":{"date-parts":[[2019,11,18]],"date-time":"2019-11-18T00:00:00Z","timestamp":1574035200000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Robotica"],"published-print":{"date-parts":[[2020,9]]},"abstract":"<jats:title>SUMMARY<\/jats:title><jats:p>In this paper, the behavioral learning of robots through spiking neural networks is studied in which the architecture of the network is based on the thalamo-cortico-thalamic circuitry of the mammalian brain. According to a variety of neurons, the Izhikevich model of single neuron is used for the representation of neuronal behaviors. One thousand and ninety spiking neurons are considered in the network. The spiking model of the proposed architecture is derived and prepared for the learning problem of robots. The reinforcement learning algorithm is based on spike-timing-dependent plasticity and dopamine release as a reward. It results in strengthening the synaptic weights of the neurons that are involved in the robot\u2019s proper performance. Sensory and motor neurons are placed in the thalamus and cortical module, respectively. The inputs of thalamo-cortico-thalamic circuitry are the signals related to distance of the target from robot, and the outputs are the velocities of actuators. The target attraction task is used as an example to validate the proposed method in which dopamine is released when the robot catches the target. Some simulation studies, as well as experimental implementation, are done on a mobile robot named Tabrizbot. Experimental studies illustrate that after successful learning, the meantime of catching target is decreased by about 36%. These prove that through the proposed method, thalamo-cortical structure could be trained successfully to learn to perform various robotic tasks.<\/jats:p>","DOI":"10.1017\/s0263574719001632","type":"journal-article","created":{"date-parts":[[2019,11,18]],"date-time":"2019-11-18T08:24:58Z","timestamp":1574065498000},"page":"1558-1575","source":"Crossref","is-referenced-by-count":9,"title":["Experimental Study of Reinforcement Learning in Mobile Robots Through Spiking Architecture of Thalamo-Cortico-Thalamic Circuitry of Mammalian Brain"],"prefix":"10.1017","volume":"38","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9967-1979","authenticated-orcid":false,"given":"Vahid","family":"Azimirad","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mohammad","family":"Fattahi Sani","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"56","published-online":{"date-parts":[[2019,11,18]]},"reference":[{"key":"S0263574719001632_ref42","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2004.1302446"},{"key":"S0263574719001632_ref36","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2011.08.060"},{"key":"S0263574719001632_ref31","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN.2010.5596542"},{"key":"S0263574719001632_ref10","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2004.832719"},{"key":"S0263574719001632_ref51","doi-asserted-by":"publisher","DOI":"10.1109\/NER.2013.6696078"},{"key":"S0263574719001632_ref20","doi-asserted-by":"publisher","DOI":"10.3389\/fnbot.2011.00001"},{"key":"S0263574719001632_ref43","doi-asserted-by":"publisher","DOI":"10.1177\/1059712308093869"},{"key":"S0263574719001632_ref33","unstructured":"[33] Nadjib Zennir, M. , Benmohammed, M. and Boudjadja, R. , \u201cSpike-Time Dependant Plasticity in a Spiking Neural Network for Robot Path Planning,\u201d AIAI Workshops (2015) pp. 2\u201313."},{"key":"S0263574719001632_ref8","doi-asserted-by":"publisher","DOI":"10.1016\/j.brainres.2014.12.023"},{"key":"S0263574719001632_ref25","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2011.02.009"},{"key":"S0263574719001632_ref60","doi-asserted-by":"crossref","first-page":"1173","DOI":"10.1016\/j.neuron.2016.02.001","article-title":"Thalamocortical projections onto behaviorally relevant neurons exhibit plasticity during adult motor learning","volume":"89","author":"Takashima","year":"2016","journal-title":"Neuron"},{"key":"S0263574719001632_ref57","doi-asserted-by":"publisher","DOI":"10.1162\/NECO_a_00521"},{"key":"S0263574719001632_ref15","first-page":"3137","article-title":"Building neurocomputational models at different levels for basal ganglia circuit","volume":"17","author":"Elibol","year":"2017","journal-title":"Istanbul Univ. J. Elect. Electron. Eng"},{"key":"S0263574719001632_ref56","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-69134-1_28"},{"key":"S0263574719001632_ref54","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2003.820440"},{"key":"S0263574719001632_ref49","doi-asserted-by":"publisher","DOI":"10.1109\/AICI.2009.448"},{"key":"S0263574719001632_ref37","doi-asserted-by":"publisher","DOI":"10.1177\/1059712312442231"},{"key":"S0263574719001632_ref17","doi-asserted-by":"publisher","DOI":"10.1177\/0278364913495721"},{"key":"S0263574719001632_ref3","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2012.11.009"},{"key":"S0263574719001632_ref58","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2009.12.084"},{"key":"S0263574719001632_ref6","doi-asserted-by":"publisher","DOI":"10.1016\/j.conb.2013.12.004"},{"key":"S0263574719001632_ref19","doi-asserted-by":"publisher","DOI":"10.3389\/fnbot.2018.00035"},{"key":"S0263574719001632_ref59","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2019.06.020"},{"key":"S0263574719001632_ref2","doi-asserted-by":"publisher","DOI":"10.1016\/j.brainres.2008.04.024"},{"key":"S0263574719001632_ref40","doi-asserted-by":"publisher","DOI":"10.1016\/j.vlsi.2016.01.002"},{"key":"S0263574719001632_ref22","unstructured":"[22] Long, L. and Fang, G. , \u201cA Review of Biologically Plausible Neuron Models for Spiking Neural Networks,\u201d In: AIAA Infotech@ Aerospace 2010 (2010) p. 3540."},{"key":"S0263574719001632_ref27","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.3486-06.2006"},{"key":"S0263574719001632_ref34","doi-asserted-by":"publisher","DOI":"10.1109\/KBEI.2017.8325015"},{"key":"S0263574719001632_ref38","doi-asserted-by":"publisher","DOI":"10.1109\/CDC.2013.6760966"},{"key":"S0263574719001632_ref35","doi-asserted-by":"publisher","DOI":"10.1142\/S0129065710002577"},{"key":"S0263574719001632_ref11","doi-asserted-by":"publisher","DOI":"10.1038\/nn.4497"},{"key":"S0263574719001632_ref7","doi-asserted-by":"publisher","DOI":"10.3758\/CABN.9.4.343"},{"key":"S0263574719001632_ref46","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2007.08.025"},{"key":"S0263574719001632_ref32","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN.2010.5596525"},{"key":"S0263574719001632_ref1","doi-asserted-by":"publisher","DOI":"10.4249\/scholarpedia.1583"},{"key":"S0263574719001632_ref28","doi-asserted-by":"publisher","DOI":"10.1007\/PL00007984"},{"key":"S0263574719001632_ref45","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2014.07.061"},{"key":"S0263574719001632_ref13","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0712231105"},{"key":"S0263574719001632_ref53","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2013.05.019"},{"key":"S0263574719001632_ref39","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCB.2012.2200674"},{"key":"S0263574719001632_ref52","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN.2016.7727892"},{"key":"S0263574719001632_ref48","doi-asserted-by":"publisher","DOI":"10.1109\/ICNC.2008.718"},{"key":"S0263574719001632_ref18","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2012.09.010"},{"key":"S0263574719001632_ref23","unstructured":"[23] Burrafato, M. and Florio, L. , \u201cA cognitive architecture based on an amygdala thalamo cortical model for developing new goals and behaviors: application in humanoid robotics,\u201d Master\u2019s thesis (Politecnico di Milano, 2012)."},{"key":"S0263574719001632_ref24","first-page":"46","article-title":"Engineering a thalamo-cortico-thalamic circuit on spinnaker: A preliminary study toward modeling sleep and wakefulness","volume":"8","author":"Bhattacharya","year":"2014","journal-title":"Front. Neural Circ"},{"key":"S0263574719001632_ref29","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4419-0340-2_15"},{"key":"S0263574719001632_ref16","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN.2015.7280750"},{"key":"S0263574719001632_ref44","doi-asserted-by":"publisher","DOI":"10.1109\/CEC.2015.7256939"},{"key":"S0263574719001632_ref41","doi-asserted-by":"publisher","DOI":"10.1109\/ICSMC.2008.4811484"},{"key":"S0263574719001632_ref14","doi-asserted-by":"publisher","DOI":"10.1093\/cercor\/bhl152"},{"key":"S0263574719001632_ref47","first-page":"221","article-title":"Neuron-based control mechanisms for a robotic arm and hand","volume":"11","author":"Singh","year":"2017","journal-title":"Int. J. Comput. Elect. Auto. Control Inf. Eng"},{"key":"S0263574719001632_ref5","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2012.00002"},{"key":"S0263574719001632_ref9","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2018.08.016"},{"key":"S0263574719001632_ref50","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2013.07.055"},{"key":"S0263574719001632_ref26","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2011.06.008"},{"key":"S0263574719001632_ref30","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-015-1848-5"},{"key":"S0263574719001632_ref4","first-page":"45","article-title":"Reinforcement learning in a bio-connectionist model based in the thalamo-cortical neural circuit","volume":"16","author":"Andr\u00e9s Chalita","year":"2016","journal-title":"Biolog. Ins. Cogn. Arch"},{"key":"S0263574719001632_ref21","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2005.06.049"},{"key":"S0263574719001632_ref12","unstructured":"[12] Sarim, M. , Schultz, T. , Kumar, M. and Jha, R. , \u201cAn Artificial Brain Mechanism to Develop a Learning Paradigm for Robot Navigation,\u201d ASME 2016 Dynamic Systems and Control Conference (American Society of Mechanical Engineers, 2016) pp. V001T03A004\u2013V001T03A004."},{"key":"S0263574719001632_ref55","doi-asserted-by":"publisher","DOI":"10.1152\/jn.1998.80.1.1"}],"container-title":["Robotica"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S0263574719001632","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,8,12]],"date-time":"2020-08-12T08:07:46Z","timestamp":1597219666000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S0263574719001632\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,18]]},"references-count":60,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2020,9]]}},"alternative-id":["S0263574719001632"],"URL":"https:\/\/doi.org\/10.1017\/s0263574719001632","relation":{},"ISSN":["0263-5747","1469-8668"],"issn-type":[{"type":"print","value":"0263-5747"},{"type":"electronic","value":"1469-8668"}],"subject":[],"published":{"date-parts":[[2019,11,18]]}}}