{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T13:53:09Z","timestamp":1774965189077,"version":"3.50.1"},"reference-count":46,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,5,22]],"date-time":"2024-05-22T00:00:00Z","timestamp":1716336000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100004672","name":"Accenture","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100004672","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>Navigation of mobile agents in unknown, unmapped environments is a critical task for achieving general autonomy. Recent advancements in combining Reinforcement Learning with Deep Neural Networks have shown promising results in addressing this challenge. However, the inherent complexity of these approaches, characterized by multi-layer networks and intricate reward objectives, limits their autonomy, increases memory footprint, and complicates adaptation to energy-efficient edge hardware. To overcome these challenges, we propose a brain-inspired method that employs a shallow architecture trained by a local learning rule for self-supervised navigation in uncharted environments. Our approach achieves performance comparable to a state-of-the-art Deep Q Network (DQN) method with respect to goal-reaching accuracy and path length, with a similar (slightly lower) number of parameters, operations, and training iterations. Notably, our self-supervised approach combines novelty-based and random walks to alleviate the need for objective reward definition and enhance agent autonomy. At the same time, the shallow architecture and local learning rule do not call for error backpropagation, decreasing the memory overhead and enabling implementation on edge neuromorphic processors. These results contribute to the potential of embodied neuromorphic agents utilizing minimal resources while effectively handling variability.<\/jats:p>","DOI":"10.3389\/frobt.2024.1372375","type":"journal-article","created":{"date-parts":[[2024,5,22]],"date-time":"2024-05-22T10:50:42Z","timestamp":1716375042000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Mapless mobile robot navigation at the edge using self-supervised cognitive map learners"],"prefix":"10.3389","volume":"11","author":[{"given":"Ioannis","family":"Polykretis","sequence":"first","affiliation":[]},{"given":"Andreea","family":"Danielescu","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2024,5,22]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"24","DOI":"10.3390\/s16010024","article-title":"Sensor fusion based model for collision free mobile robot navigation","volume":"16","author":"Almasri","year":"2015","journal-title":"Sensors"},{"key":"B2","doi-asserted-by":"publisher","first-page":"5021","DOI":"10.1109\/jsen.2016.2553126","article-title":"Trajectory planning and collision avoidance algorithm for mobile robotics system","volume":"16","author":"Almasri","year":"2016","journal-title":"IEEE Sensors J."},{"key":"B3","first-page":"609","article-title":"Vector matrix multiplication using crossbar arrays: a comparative analysis","author":"Assaf","year":"2018"},{"key":"B4","doi-asserted-by":"publisher","first-page":"89","DOI":"10.1080\/23746149.2016.1259585","article-title":"Neuromorphic computing using non-volatile memory","volume":"2","author":"Burr","year":"2017","journal-title":"Adv. Phys. X"},{"key":"B5","first-page":"497","article-title":"Memory system characterization of deep learning workloads","author":"Chishti","year":"2019"},{"key":"B6","volume-title":"Visual wake words dataset","author":"Chowdhery","year":"2019"},{"key":"B7","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1109\/mm.2018.112130359","article-title":"Loihi: a neuromorphic manycore processor with on-chip learning","volume":"38","author":"Davies","year":"2018","journal-title":"IEEE Micro"},{"key":"B8","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1007\/978-3-031-28715-2_7","article-title":"Deep reinforcement learning for autonomous mobile robot navigation","volume":"1093","author":"de Jes\u00fas Plasencia-Salgueiro","year":"2023","journal-title":"Artif. Intell. Robotics Aut. Syst. Appl."},{"key":"B9","doi-asserted-by":"publisher","first-page":"756","DOI":"10.1109\/tcsvt.2022.3203974","article-title":"Monocular camera-based complex obstacle avoidance via efficient deep reinforcement learning","volume":"33","author":"Ding","year":"2022","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"B10","first-page":"398","article-title":"What\u2019s special about spatial? database requirements for vehicle navigation in geographic space","author":"Egenhofer","year":"1993"},{"key":"B11","doi-asserted-by":"publisher","first-page":"652","DOI":"10.1109\/jproc.2014.2304638","article-title":"The spinnaker project","volume":"102","author":"Furber","year":"2014","journal-title":"Proc. IEEE"},{"key":"B12","first-page":"1861","article-title":"Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor","author":"Haarnoja","year":"2018"},{"key":"B13","unstructured":"Stable baselines\n            HillA.\n            RaffinA.\n            ErnestusM.\n            GleaveA.\n            KanervistoA.\n            TraoreR.\n          2018"},{"key":"B14","doi-asserted-by":"publisher","first-page":"1416","DOI":"10.1126\/science.1138353","article-title":"From swimming to walking with a salamander robot driven by a spinal cord model","volume":"315","author":"Ijspeert","year":"2007","journal-title":"science"},{"key":"B15","doi-asserted-by":"publisher","first-page":"1312","DOI":"10.1109\/lra.2021.3057023","article-title":"Badgr: an autonomous self-supervised learning-based navigation system","volume":"6","author":"Kahn","year":"2021","journal-title":"IEEE Robotics Automation Lett."},{"key":"B16","first-page":"5129","article-title":"Self-supervised deep reinforcement learning with generalized computation graphs for robot navigation","author":"Kahn","year":"2018"},{"key":"B17","volume-title":"Modularizing and assembling cognitive map learners via hyperdimensional computing","author":"McDonald","year":"2023"},{"key":"B18","doi-asserted-by":"publisher","first-page":"668","DOI":"10.1126\/science.1254642","article-title":"A million spiking-neuron integrated circuit with a scalable communication network and interface","volume":"345","author":"Merolla","year":"2014","journal-title":"Science"},{"key":"B19","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1016\/s1389-0417(03)00007-x","article-title":"Map-based navigation in mobile robots:","volume":"4","author":"Meyer","year":"2003","journal-title":"Cognitive Syst. Res."},{"key":"B20","doi-asserted-by":"publisher","first-page":"39","DOI":"10.5772\/5618","article-title":"Webots: professional mobile robot simulation","volume":"1","author":"Michel","year":"2004","journal-title":"J. Adv. Robotics Syst."},{"key":"B21","first-page":"1928","article-title":"Asynchronous methods for deep reinforcement learning","author":"Mnih","year":"2016"},{"key":"B22","volume-title":"Playing atari with deep reinforcement learning","author":"Mnih","year":"2013"},{"key":"B23","doi-asserted-by":"publisher","first-page":"106","DOI":"10.1109\/tbcas.2017.2759700","article-title":"A scalable multicore architecture with heterogeneous memory structures for dynamic neuromorphic asynchronous processors (dynaps)","volume":"12","author":"Moradi","year":"2017","journal-title":"IEEE Trans. Biomed. circuits Syst."},{"key":"B24","doi-asserted-by":"publisher","first-page":"324","DOI":"10.3389\/fnins.2017.00324","article-title":"Event-driven random back-propagation: enabling neuromorphic deep learning machines","volume":"11","author":"Neftci","year":"2017","journal-title":"Front. Neurosci."},{"key":"B25","doi-asserted-by":"publisher","first-page":"610","DOI":"10.1109\/lra.2019.2891991","article-title":"Deep reinforcement learning robot for search and rescue applications: exploration in unknown cluttered environments","volume":"4","author":"Niroui","year":"2019","journal-title":"IEEE Robotics Automation Lett."},{"key":"B26","doi-asserted-by":"publisher","first-page":"795876","DOI":"10.3389\/fnins.2022.795876","article-title":"The brainscales-2 accelerated neuromorphic system with hybrid plasticity","volume":"16","author":"Pehle","year":"2022","journal-title":"Front. Neurosci."},{"key":"B27","doi-asserted-by":"publisher","first-page":"014013","DOI":"10.1088\/2634-4386\/acc204","article-title":"Bioinspired smooth neuromorphic control for robotic arms","volume":"3","author":"Polykretis","year":"2023","journal-title":"Neuromorphic Comput. Eng."},{"key":"B28","first-page":"1","article-title":"An astrocyte-modulated neuromorphic central pattern generator for hexapod robot locomotion on intel\u2019s loihi","author":"Polykretis","year":"2020"},{"key":"B29","doi-asserted-by":"crossref","DOI":"10.21203\/rs.3.rs-701752\/v1","volume-title":"The backpropagation algorithm implemented on spiking neuromorphic hardware","author":"Renner","year":"2021"},{"key":"B30","doi-asserted-by":"publisher","first-page":"69","DOI":"10.1016\/s0020-0255(02)00224-4","article-title":"A behavior-based architecture for autonomous underwater exploration","volume":"145","author":"Rosenblatt","year":"2002","journal-title":"Inf. Sci."},{"key":"B31","doi-asserted-by":"publisher","first-page":"533","DOI":"10.1038\/323533a0","article-title":"Learning representations by back-propagating errors","volume":"323","author":"Rumelhart","year":"1986","journal-title":"nature"},{"key":"B32","first-page":"1889","article-title":"Trust region policy optimization","author":"Schulman","year":"2015"},{"key":"B33","volume-title":"Proximal policy optimization algorithms","author":"Schulman","year":"2017"},{"key":"B34","doi-asserted-by":"publisher","first-page":"461","DOI":"10.1007\/s10846-017-0680-9","article-title":"Towards autonomous planetary exploration","volume":"93","author":"Schuster","year":"2019","journal-title":"J. Intelligent Robotic Syst."},{"key":"B35","first-page":"71","article-title":"Speech2spikes: efficient audio encoding pipeline for real-time neuromorphic systems","author":"Stewart","year":"2023"},{"key":"B36","doi-asserted-by":"publisher","first-page":"2344","DOI":"10.1038\/s41467-024-46586-0","article-title":"Local prediction-learning in high-dimensional spaces enables neural networks to plan","volume":"15","author":"St\u00f6ckl","year":"2022","journal-title":"Nat. Commun."},{"key":"B37","doi-asserted-by":"publisher","first-page":"80","DOI":"10.1038\/nature06932","article-title":"The missing memristor found","volume":"453","author":"Strukov","year":"2008","journal-title":"nature"},{"key":"B38","first-page":"31","article-title":"Virtual-to-real deep reinforcement learning: continuous control of mobile robots for mapless navigation","author":"Tai","year":"2017"},{"key":"B39","first-page":"182","article-title":"Autonomous robot navigation based on fuzzy sensor fusion and reinforcement learning","author":"Tan","year":"2002"},{"key":"B40","first-page":"6090","article-title":"Reinforcement co-learning of deep and spiking neural networks for energy-efficient mapless navigation with neuromorphic hardware","author":"Tang","year":"2020"},{"key":"B41","first-page":"8114","article-title":"Novelty search in representational space for sample efficient exploration","volume":"33","author":"Tao","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"B42","doi-asserted-by":"crossref","DOI":"10.15607\/RSS.2020.XVI.020","volume-title":"Event-driven visual-tactile sensing and learning for robots","author":"Taunyazov","year":"2020"},{"key":"B43","doi-asserted-by":"publisher","first-page":"356","DOI":"10.5573\/jsts.2014.14.3.356","article-title":"New memristor-based crossbar array architecture with 50-% area reduction and 48-% power saving for matrix-vector multiplication of analog neuromorphic computing","volume":"14","author":"Truong","year":"2014","journal-title":"J. Semicond. Technol. Sci."},{"key":"B44","doi-asserted-by":"publisher","first-page":"320","DOI":"10.1016\/j.snb.2012.01.074","article-title":"Chemical gas sensor drift compensation using classifier ensembles","volume":"166","author":"Vergara","year":"2012","journal-title":"Sensors Actuators B Chem."},{"key":"B45","doi-asserted-by":"publisher","first-page":"935","DOI":"10.1109\/jproc.2020.3045625","article-title":"Brain-inspired learning on neuromorphic substrates","volume":"109","author":"Zenke","year":"2021","journal-title":"Proc. IEEE"},{"key":"B46","first-page":"3357","article-title":"Target-driven visual navigation in indoor scenes using deep reinforcement learning","author":"Zhu","year":"2017"}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2024.1372375\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,22]],"date-time":"2024-05-22T10:50:49Z","timestamp":1716375049000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2024.1372375\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,22]]},"references-count":46,"alternative-id":["10.3389\/frobt.2024.1372375"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2024.1372375","relation":{},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,22]]},"article-number":"1372375"}}