{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,26]],"date-time":"2026-10-26T22:23:35Z","timestamp":1793053415463,"version":"4.5.0"},"reference-count":24,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,9,23]],"date-time":"2024-09-23T00:00:00Z","timestamp":1727049600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurosci."],"abstract":"<jats:p>Mobile, low-cost, and energy-aware operation of Artificial Intelligence (AI) computations in smart circuits and autonomous robots will play an important role in the next industrial leap in intelligent automation and assistive devices. Neuromorphic hardware with spiking neural network (SNN) architecture utilizes insights from biological phenomena to offer encouraging solutions. Previous studies have proposed reinforcement learning (RL) models for SNN responses in the rat hippocampus to an environment where rewards depend on the context. The scale of these models matches the scope and capacity of small embedded systems in the framework of Internet-of-Bodies (IoB), autonomous sensor nodes, and other edge applications. Addressing energy-efficient artificial learning problems in such systems enables smart micro-systems with edge intelligence. A novel bio-inspired RL system architecture is presented in this work, leading to significant energy consumption benefits without foregoing real-time autonomous processing and accuracy requirements of the context-dependent task. The hardware architecture successfully models features analogous to synaptic tagging, changes in the exploration schemes, synapse saturation, and spatially localized task-based activation observed in the brain. The design has been synthesized, simulated, and tested on Intel MAX10 Field-Programmable Gate Array (FPGA). The problem-based bio-inspired approach to SNN edge architectural design results in 25X reduction in average power compared to the state-of-the-art for a test with real-time context learning and 30 trials. Furthermore, 940x lower energy consumption is achieved due to improvement in the execution time.<\/jats:p>","DOI":"10.3389\/fnins.2024.1431222","type":"journal-article","created":{"date-parts":[[2024,9,23]],"date-time":"2024-09-23T05:10:16Z","timestamp":1727068216000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":13,"title":["Energy-aware bio-inspired spiking reinforcement learning system architecture for real-time autonomous edge applications"],"prefix":"10.3389","volume":"18","author":[{"given":"Joshua Ifeanyi","family":"Okonkwo","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mohamed S.","family":"Abdelfattah","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peyman","family":"Mirtaheri","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ali","family":"Muhtaroglu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2024,9,23]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"498","DOI":"10.1109\/JETCAS.2020.3031040","article-title":"Digital multiplier-less spiking neural network architecture of reinforcement learning in a context-dependent task","volume":"10","author":"Asgari","year":"2020","journal-title":"IEEE J. Emerg. Select. Top. Circ. Syst"},{"key":"B2","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1101\/103135","article-title":"Pure correlates of exploration and exploitation in the human brain","volume":"18","author":"Blanchard","year":"2017","journal-title":"Cogn. Affect. Behav. Neurosci"},{"key":"B3","unstructured":"Edge AI: The Cloud-Free Future Is Here\n          \n          2023"},{"key":"B4","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1016\/j.neuron.2004.09.007","article-title":"Spike timing-dependent plasticity of neural circuits","volume":"44","author":"Dan","year":"2004","journal-title":"Neuron"},{"key":"B5","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1109\/MM.2018.112130359","article-title":"Loihi: a neuromorphic manycore processor with on-chip learning","volume":"38","author":"Davies","year":"2018","journal-title":"IEEE Micro"},{"key":"B6","doi-asserted-by":"publisher","first-page":"436","DOI":"10.1016\/j.neuroimage.2003.08.041","article-title":"Learned regulation of spatially localized brain activation using real-time fmri","volume":"21","author":"deCharms","year":"2004","journal-title":"Neuroimage"},{"key":"B7","doi-asserted-by":"publisher","first-page":"eabb0184","DOI":"10.1126\/science.abb0184","article-title":"Neural mechanisms resolving exploitation-exploration dilemmas in the medial prefrontal cortex","volume":"369","author":"Domenech","year":"2020","journal-title":"Science"},{"key":"B8","doi-asserted-by":"publisher","first-page":"795","DOI":"10.1109\/TBCAS.2019.2925454","article-title":"Discrimination of EMG signals using a neuromorphic implementation of a spiking neural network","volume":"13","author":"Donati","year":"2019","journal-title":"IEEE Trans. Biomed. Circuits Syst"},{"key":"B9","doi-asserted-by":"publisher","first-page":"1582","DOI":"10.1109\/TCSII.2019.2890846","article-title":"A low-cost high-speed neuromorphic hardware based on spiking neural network","volume":"66","author":"Farsa","year":"2019","journal-title":"IEEE Transact. Circ. Syst. II Exp. Briefs"},{"key":"B10","doi-asserted-by":"publisher","first-page":"533","DOI":"10.1038\/385533a0","article-title":"Synaptic tagging and long-term potentiation","volume":"385","author":"Frey","year":"1997","journal-title":"Nature"},{"key":"B11","volume-title":"Spiking Neuron Models: Single Neurons, Populations, Plasticity","author":"Gerstner","year":"2012"},{"key":"B12","doi-asserted-by":"publisher","first-page":"500","DOI":"10.1113\/jphysiol.1952.sp004764","article-title":"A quantitative description of mem-brane current and application to conduction and excitation in nerve","volume":"117","author":"Hodgkin","year":"1952","journal-title":"J. Physiol"},{"key":"B13","doi-asserted-by":"publisher","first-page":"1063","DOI":"10.1109\/TNN.2004.832719","article-title":"Which model to use for cortical spiking neurons?","volume":"15","author":"Izhikevich","year":"2004","journal-title":"IEEE Transact. Neural Netw"},{"key":"B14","doi-asserted-by":"publisher","first-page":"255","DOI":"10.1038\/s41586-021-04362-w","article-title":"Brain-inspired computing needs a master plan","volume":"604","author":"Mechonic","year":"2022","journal-title":"Nature"},{"key":"B15","doi-asserted-by":"publisher","first-page":"e20147","DOI":"10.7554\/eLife.20147","article-title":"A saturation hypothesis to explain both enhanced and impaired learning with enhanced plasticity","volume":"6","author":"Nguyen-Vu","year":"2017","journal-title":"Elife"},{"key":"B16","volume-title":"Images of Mind","author":"Posner","year":"1994"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.1109\/ISCAS58744.2024.10558211","article-title":"A reduced spiking neural network architecture for energy efficient context-dependent reinforcement learning tasks","author":"Rasheed","year":"2024","journal-title":"IEEE Int. Symp. Circuits Syst."},{"key":"B18","doi-asserted-by":"publisher","first-page":"178","DOI":"10.3389\/fnsys.2014.00178","article-title":"A model of hippocampal spiking responses to items during learning of a context-dependent task","volume":"8","author":"Raudies","year":"2014","journal-title":"Front. Syst. Neurosci"},{"key":"B19","doi-asserted-by":"publisher","first-page":"2991","DOI":"10.1109\/TCSI.2012.2206463","article-title":"Biologically inspired spiking neurons: piecewise linear models and digital implementation","volume":"59","author":"Soleimani","year":"2012","journal-title":"IEEE Transact. Circ. Syst. I Regul. Pap"},{"key":"B20","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton","year":"2018"},{"key":"B21","article-title":"IBM research report 1014 RJ10502 (ALM1211\u2013004)","author":"Wong","year":"2012","journal-title":"IBM Comp. Sci"},{"key":"B22","doi-asserted-by":"publisher","first-page":"148","DOI":"10.1109\/TNNLS.2019.2899936","article-title":"Scalable digital neuromorphic architecture for large-scale biophysically meaningful neural network with multi-compartment neurons","volume":"31","author":"Yang","year":"2020","journal-title":"IEEE Transact. Neural Netw. Learn. Syst"},{"key":"B23","doi-asserted-by":"publisher","first-page":"186","DOI":"10.1109\/TBCAS.2023.3316968","article-title":"NADOL: neuromorphic architecture for spike-driven online learning by dendrites","volume":"18","author":"Yang","year":"2024","journal-title":"IEEE Trans. Biomed. Circuits Syst"},{"key":"B24","doi-asserted-by":"publisher","first-page":"2801","DOI":"10.1109\/TNNLS.2020.3045492","article-title":"BiCoSS: toward large-scale cognition brain with multigranular neuromorphic architecture","volume":"33","author":"Yang","year":"2022","journal-title":"IEEE Transact. Neural Netw. Learn. Syst"}],"container-title":["Frontiers in Neuroscience"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnins.2024.1431222\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,23]],"date-time":"2024-09-23T05:10:19Z","timestamp":1727068219000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnins.2024.1431222\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,23]]},"references-count":24,"alternative-id":["10.3389\/fnins.2024.1431222"],"URL":"https:\/\/doi.org\/10.3389\/fnins.2024.1431222","relation":{},"ISSN":["1662-453X"],"issn-type":[{"value":"1662-453X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,23]]},"article-number":"1431222"}}