{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T18:46:29Z","timestamp":1775069189360,"version":"3.50.1"},"reference-count":114,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,9,9]],"date-time":"2024-09-09T00:00:00Z","timestamp":1725840000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001663","name":"Volkswagen Foundation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100001663","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Comput. Neurosci."],"abstract":"<jats:p>Bees are among the master navigators of the insect world. Despite impressive advances in robot navigation research, the performance of these insects is still unrivaled by any artificial system in terms of training efficiency and generalization capabilities, particularly considering the limited computational capacity. On the other hand, computational principles underlying these extraordinary feats are still only partially understood. The theoretical framework of reinforcement learning (RL) provides an ideal focal point to bring the two fields together for mutual benefit. In particular, we analyze and compare representations of space in robot and insect navigation models through the lens of RL, as the efficiency of insect navigation is likely rooted in an efficient and robust internal representation, linking retinotopic (egocentric) visual input with the geometry of the environment. While RL has long been at the core of robot navigation research, current computational theories of insect navigation are not commonly formulated within this framework, but largely as an associative learning process implemented in the insect brain, especially in the mushroom body (MB). Here we propose specific hypothetical components of the MB circuit that would enable the implementation of a certain class of relatively simple RL algorithms, capable of integrating distinct components of a navigation task, reminiscent of hierarchical RL models used in robot navigation. We discuss how current models of insect and robot navigation are exploring representations beyond classical, complete map-like representations, with spatial information being embedded in the respective latent representations to varying degrees.<\/jats:p>","DOI":"10.3389\/fncom.2024.1460006","type":"journal-article","created":{"date-parts":[[2024,9,9]],"date-time":"2024-09-09T04:53:14Z","timestamp":1725857594000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Reinforcement learning as a robotics-inspired framework for insect navigation: from spatial representations to neural implementation"],"prefix":"10.3389","volume":"18","author":[{"given":"Stephan","family":"Lochner","sequence":"first","affiliation":[]},{"given":"Daniel","family":"Honerkamp","sequence":"additional","affiliation":[]},{"given":"Abhinav","family":"Valada","sequence":"additional","affiliation":[]},{"given":"Andrew D.","family":"Straw","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2024,9,9]]},"reference":[{"key":"B1","article-title":"Deep variational information bottleneck","author":"Alemi","year":"2019","journal-title":"arXiv preprint arXiv:1612.00410"},{"key":"B2","doi-asserted-by":"publisher","first-page":"e1004683","DOI":"10.1371\/journal.pcbi.1004683","article-title":"Using an insect mushroom body circuit to encode route memory in complex natural environments","volume":"12","author":"Ardin","year":"2016","journal-title":"PLoS Comput. Biol"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.3389\/fnbot.2017.00012","article-title":"Motor-skill learning in an insect inspired neuro-computational control system","author":"Arena","year":"2017","journal-title":"Front. Neurorobot"},{"key":"B4","doi-asserted-by":"publisher","first-page":"e04580","DOI":"10.7554\/eLife.04580","article-title":"Mushroom body output neurons encode valence and guide memory-based action selection in Drosophila","volume":"3","author":"Aso","year":"2014","journal-title":"Elife"},{"key":"B5","doi-asserted-by":"publisher","first-page":"e1002336","DOI":"10.1371\/journal.pcbi.1002336","article-title":"A model of ant route navigation driven by scene familiarity","volume":"8","author":"Baddeley","year":"2012","journal-title":"PLoS Comput. Biol"},{"key":"B6","doi-asserted-by":"publisher","first-page":"0025","DOI":"10.34133\/icomputing.0025","article-title":"Evolutionary reinforcement learning: a survey","volume":"2","author":"Bai","year":"2023","journal-title":"Intell. Comput"},{"key":"B7","article-title":"Deep reinforcement learning on a budget: 3D control and reasoning without a supercomputer","author":"Beeching","year":"2019","journal-title":"arXiv preprint arXiv:1904.01806"},{"key":"B8","doi-asserted-by":"publisher","first-page":"2569","DOI":"10.1038\/s41467-021-22592-4","article-title":"Learning with reinforcement prediction errors in a model of the Drosophila mushroom body","volume":"12","author":"Bennett","year":"2021","journal-title":"Nat. Commun"},{"key":"B9","doi-asserted-by":"publisher","first-page":"103981","DOI":"10.1016\/j.trc.2022.103981","article-title":"A deep reinforcement learning approach for solving the Traveling Salesman Problem with Drone","volume":"148","author":"Bogyrbayeva","year":"2023","journal-title":"Transport. Res. Part C"},{"key":"B10","article-title":"Exploration by random network distillation","author":"Burda","year":"2018","journal-title":"arXiv preprint arXiv:1810.12894"},{"key":"B11","doi-asserted-by":"publisher","first-page":"113","DOI":"10.1038\/nature12063","article-title":"Random convergence of olfactory inputs in the drosophila mushroom body","volume":"497","author":"Caron","year":"2013","journal-title":"Nature"},{"key":"B12","doi-asserted-by":"publisher","first-page":"521","DOI":"10.1007\/BF00605469","article-title":"Landmark learning in bees","volume":"151","author":"Cartwright","year":"1983","journal-title":"J. Compar. Physiol"},{"key":"B13","article-title":"Learning to explore using active neural slam","author":"Chaplot","year":"2020","journal-title":"arXiv preprint arXiv:2004.05155"},{"key":"B14","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1016\/j.conb.2017.12.002","article-title":"Do the right thing: neural network mechanisms of memory formation, expression and update in drosophila","volume":"49","author":"Cognigni","year":"2018","journal-title":"Curr. Opin. Neurobiol"},{"key":"B15","doi-asserted-by":"publisher","first-page":"jeb245278","DOI":"10.1242\/jeb.245278","article-title":"An \u2018instinct for learning': the learning flights and walks of bees, wasps and ants from the 1850s to now","volume":"226","author":"Collett","year":"2023","journal-title":"J. Exper. Biol"},{"key":"B16","doi-asserted-by":"publisher","first-page":"613","DOI":"10.1162\/neco.1993.5.4.613","article-title":"Improving generalization for temporal difference learning: the successor representation","volume":"5","author":"Dayan","year":"1993","journal-title":"Neural Comput"},{"key":"B17","doi-asserted-by":"publisher","first-page":"62","DOI":"10.1016\/j.shpsa.2022.12.008","article-title":"The cognitive map debate in insects: a historical perspective on what is at stake","volume":"98","author":"Dhein","year":"2023","journal-title":"Stud. Hist. Philos. Sci"},{"key":"B18","doi-asserted-by":"publisher","first-page":"397","DOI":"10.1038\/nature09633","article-title":"Preplay of future place cell sequences by hippocampal cellular assemblies","volume":"469","author":"Dragoi","year":"2011","journal-title":"Nature"},{"key":"B19","doi-asserted-by":"publisher","first-page":"362","DOI":"10.1002\/cne.10355","article-title":"Segregation of visual input to the mushroom bodies in the honeybee (Apis mellifera)","volume":"451","author":"Ehmer","year":"2002","journal-title":"J. Compar. Neurol"},{"key":"B20","doi-asserted-by":"crossref","first-page":"1691","DOI":"10.1109\/ICRA.2012.6225199","article-title":"\u201cAn evaluation of the rgb-d slam system,\u201d","volume-title":"2012 IEEE International Conference on Robotics and Automation","author":"Endres","year":"2012"},{"key":"B21","doi-asserted-by":"publisher","first-page":"834","DOI":"10.1007\/978-3-319-10605-2_54","article-title":"\u201cLSD-SLAM: large-scale direct monocular SLAM,\u201d","author":"Engel","year":"2014","journal-title":"Computer Vision-ECCV 2014"},{"key":"B22","doi-asserted-by":"publisher","first-page":"544","DOI":"10.1038\/s41593-020-0607-9","article-title":"Recurrent architecture for adaptive regulation of learning in the insect brain","volume":"23","author":"Eschbach","year":"2020","journal-title":"Nat. Neurosci"},{"key":"B23","doi-asserted-by":"publisher","first-page":"96","DOI":"10.1016\/j.neunet.2016.11.002","article-title":"A computational model of conditioning inspired by Drosophila olfactory system","volume":"87","author":"Faghihi","year":"2017","journal-title":"Neural Netw"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00063","article-title":"\u201cScene memory transformer for embodied agents in long-horizon tasks,\u201d","author":"Fang","year":"2019","journal-title":"2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)"},{"key":"B25","doi-asserted-by":"publisher","first-page":"240","DOI":"10.1038\/nature21716","article-title":"Re-evaluation of learned information in Drosophila","volume":"544","author":"Felsenberg","year":"2017","journal-title":"Nature"},{"key":"B26","doi-asserted-by":"publisher","first-page":"428","DOI":"10.1038\/s41583-024-00817-x","article-title":"Remapping revisited: How the hippocampus represents different spaces","volume":"25","author":"Fenton","year":"2024","journal-title":"Nat. Rev. Neurosci"},{"key":"B27","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1007\/s10462-012-9365-8","article-title":"Visual simultaneous localization and mapping: a survey","volume":"43","author":"Fuentes-Pacheco","year":"2015","journal-title":"Artif. Intell. Rev"},{"key":"B28","doi-asserted-by":"publisher","first-page":"252","DOI":"10.1016\/B978-1-55860-377-6.50039-6","article-title":"\u201cAnt-Q: a reinforcement learning approach to the traveling salesman problem,\u201d","author":"Gambardella","year":"1995","journal-title":"Machine Learning Proceedings"},{"key":"B29","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s42003-022-03075-8","article-title":"Visual learning in a virtual reality environment upregulates immediate early gene expression in the mushroom bodies of honey bees","volume":"5","author":"Geng","year":"2022","journal-title":"Commun. Biol"},{"key":"B30","doi-asserted-by":"publisher","first-page":"930","DOI":"10.1038\/35073582","article-title":"The concepts of \u2018sameness' and \u2018difference' in an insect","volume":"410","author":"Giurfa","year":"2001","journal-title":"Nature"},{"key":"B31","doi-asserted-by":"publisher","first-page":"e1011480","DOI":"10.1371\/journal.pcbi.1011480","article-title":"Emergent spatial goals in an integrative model of the insect central complex","volume":"19","author":"Goulard","year":"2023","journal-title":"PLoS Comput. Biol"},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA57147.2024.10610112","article-title":"Collaborative dynamic 3d scene graphs for automated driving","author":"Greve","year":"2023","journal-title":"arXiv preprint arXiv:2309.06635"},{"key":"B33","article-title":"ConceptGraphs: open-vocabulary 3D scene graphs for perception and planning","author":"Gu","year":"2023","journal-title":"arXiv preprint arXiv:2309.16650"},{"key":"B34","article-title":"Unifying map and landmark based representations for visual navigation","author":"Gupta","year":"2017","journal-title":"arXiv preprint arXiv:1712.08125"},{"key":"B35","article-title":"Cognitive mapping and planning for visual navigation","author":"Gupta","year":"2019","journal-title":"arXiv preprint arXiv:1702.03920"},{"key":"B36","doi-asserted-by":"publisher","first-page":"1117","DOI":"10.1177\/0278364908096316","article-title":"3D perception and environment map generation for humanoid robot navigation","volume":"27","author":"Gutmann","year":"2008","journal-title":"Int. J. Rob. Res"},{"key":"B37","article-title":"Latent space policies for hierarchical reinforcement learning","author":"Haarnoja","year":"2018","journal-title":"arXiv preprint arXiv:1804.02808"},{"key":"B38","article-title":"Deep hierarchical planning from pixels","author":"Hafner","year":"2022","journal-title":"arXiv preprint arXiv:2206.04114"},{"key":"B39","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00884","article-title":"\u201cMapNet: an allocentric spatial memory for mapping environments,\u201d","author":"Henriques","year":"2018","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B40","doi-asserted-by":"publisher","first-page":"2824","DOI":"10.1073\/pnas.1721668115","article-title":"Optimal multiguidance integration in insect navigation","volume":"115","author":"Hoinville","year":"2018","journal-title":"Proc. Nat. Acad. Sci"},{"key":"B41","article-title":"Language-grounded dynamic scene graphs for interactive object search with mobile manipulation","author":"Honerkamp","year":"2024","journal-title":"arXiv preprint arXiv:2403.08605"},{"key":"B42","doi-asserted-by":"publisher","first-page":"2123","DOI":"10.1162\/neco.2009.03-08-733","article-title":"Fast and robust learning by reinforcement signals: explorations in the insect brain","volume":"21","author":"Huerta","year":"2009","journal-title":"Neural Comput"},{"key":"B43","doi-asserted-by":"publisher","first-page":"1601","DOI":"10.1162\/089976604774201613","article-title":"Learning classification in the olfactory system of insects","volume":"16","author":"Huerta","year":"2004","journal-title":"Neural Comput"},{"key":"B44","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2022.XVIII.050","article-title":"Hydra: a real-time spatial perception system for 3D scene graph construction and optimization","author":"Hughes","year":"2022","journal-title":"arXiv preprint arXiv:2201.13360"},{"key":"B45","doi-asserted-by":"publisher","first-page":"e66039","DOI":"10.7554\/eLife.66039","article-title":"A connectome of the Drosophila central complex reveals network motifs suitable for flexible navigation and context-dependent action selection","volume":"10","author":"Hulse","year":"2021","journal-title":"Elife"},{"key":"B46","doi-asserted-by":"publisher","first-page":"10693","DOI":"10.1073\/pnas.1201880109","article-title":"From chemotaxis to the cognitive map: the function of olfaction","volume":"109","author":"Jacobs","year":"2012","journal-title":"Proc. Nat. Acad. Sci"},{"key":"B47","doi-asserted-by":"publisher","first-page":"jeb185306","DOI":"10.1242\/jeb.185306","article-title":"The choreography of learning walks in the Australian jack jumper ant Myrmecia croslandi","volume":"221","author":"Jayatilaka","year":"2018","journal-title":"J. Exper. Biol"},{"key":"B48","doi-asserted-by":"publisher","first-page":"108640","DOI":"10.1016\/j.isci.2023.108640","article-title":"Prediction error drives associative learning and conditioned behavior in a spiking model of Drosophila larva","volume":"27","author":"J\u00fcrgensen","year":"2024","journal-title":"iScience"},{"key":"B49","first-page":"14291","article-title":"\u201cDeep inverse q-learning with constraints,\u201d","volume-title":"Advances in Neural Information Processing Systems","author":"Kalweit","year":"2020"},{"key":"B50","article-title":"NeuRL: closed-form inverse reinforcement learning for neural decoding","author":"Kalweit","year":"2022","journal-title":"arXiv preprint arXiv:2204.04733"},{"key":"B51","doi-asserted-by":"publisher","first-page":"20190103","DOI":"10.1098\/rsif.2019.0103","article-title":"Bumblebees learn foraging routes through exploitation-exploration cycles","volume":"16","author":"Kembro","year":"2019","journal-title":"J. R. Soc. Interface"},{"key":"B52","doi-asserted-by":"publisher","first-page":"744","DOI":"10.1038\/s41583-022-00642-0","article-title":"Attractor and integrator networks in the brain","volume":"23","author":"Khona","year":"2022","journal-title":"Nat. Rev. Neurosci"},{"key":"B53","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/ISMAR.2007.4538852","article-title":"\u201cParallel tracking and mapping for small AR workspaces,\u201d","volume-title":"2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality","author":"Klein","year":"2007"},{"key":"B54","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2009.V.020","article-title":"\u201cView-based maps,\u201d","author":"Konolige","year":"2009","journal-title":"Robotics: Science and Systems V"},{"key":"B55","doi-asserted-by":"publisher","first-page":"21127","DOI":"10.1038\/s41598-021-00630-x","article-title":"Motion cues from the background influence associative color learning of honey bees in a virtual-reality scenario","volume":"11","author":"Lafon","year":"2021","journal-title":"Sci. Rep"},{"key":"B56","doi-asserted-by":"publisher","first-page":"690","DOI":"10.3389\/fpsyg.2019.00690","article-title":"The central complex as a potential substrate for vector based navigation","volume":"10","author":"Le Mo\u00ebl","year":"2019","journal-title":"Front. Psychol"},{"key":"B57","doi-asserted-by":"publisher","first-page":"417","DOI":"10.1162\/089976699300016719","article-title":"Independent component analysis using an extended infomax algorithm for mixed subgaussian and supergaussian sources","volume":"11","author":"Lee","year":"1999","journal-title":"Neural Comput"},{"key":"B58","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1098\/rsbl.2011.0661","article-title":"Bees do not use nearest-neighbour rules for optimization of multi-location routes","volume":"8","author":"Lihoreau","year":"2011","journal-title":"Biol. Lett"},{"key":"B59","doi-asserted-by":"publisher","first-page":"744","DOI":"10.1086\/657042","article-title":"Travel optimization by foraging bumblebees through readjustments of traplines after discovery of new feeding locations","volume":"176","author":"Lihoreau","year":"2010","journal-title":"Am. Nat"},{"key":"B60","doi-asserted-by":"publisher","first-page":"e1001392","DOI":"10.1371\/journal.pbio.1001392","article-title":"Radar tracking and motion-sensitive cameras on flowers reveal the development of pollinator multi-destination routes over large spatial scales","volume":"10","author":"Lihoreau","year":"2012","journal-title":"PLoS Biol"},{"key":"B61","doi-asserted-by":"publisher","first-page":"512","DOI":"10.1038\/nature11304","article-title":"A subset of dopamine neurons signals reward for odour memory in Drosophila","volume":"488","author":"Liu","year":"2012","journal-title":"Nature"},{"key":"B62","doi-asserted-by":"publisher","first-page":"909","DOI":"10.1162\/NECO_a_00097","article-title":"An Infomax algorithm can perform both familiarity discrimination and feature extraction in a single network","volume":"23","author":"Lulham","year":"2011","journal-title":"Neural Comput"},{"key":"B63","doi-asserted-by":"publisher","first-page":"92","DOI":"10.1038\/s41586-021-04067-0","article-title":"Building an allocentric travelling direction signal via vector computation","volume":"601","author":"Lyu","year":"2022","journal-title":"Nature"},{"key":"B64","doi-asserted-by":"publisher","first-page":"4613","DOI":"10.1038\/s41467-022-32247-7","article-title":"A neural circuit for wind-guided olfactory navigation","volume":"13","author":"Matheson","year":"2022","journal-title":"Nat. Commun"},{"key":"B65","article-title":"Gaussian splatting slam","author":"Matsuki","year":"2023","journal-title":"arXiv preprint arXiv:2312.06741"},{"key":"B66","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"B67","doi-asserted-by":"publisher","first-page":"309","DOI":"10.1098\/rstb.1982.0086","article-title":"The brain of the honeybee Apis mellifera. I. The connections and spatial organization of the mushroom bodies","volume":"298","author":"Mobbs","year":"1997","journal-title":"Philos. Trans. R. Soc. London"},{"key":"B68","doi-asserted-by":"publisher","first-page":"1255","DOI":"10.1109\/TRO.2017.2705103","article-title":"ORB-SLAM2: an open-source SLAM system for monocular, stereo, and RGB-D cameras","volume":"33","author":"Mur-Artal","year":"2017","journal-title":"IEEE Trans. Robot"},{"key":"B69","article-title":"Data-efficient hierarchical reinforcement learning","author":"Nachum","year":"2018","journal-title":"arXiv preprint arXiv:1805.08296"},{"key":"B70","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1016\/0006-8993(71)90358-1","article-title":"The hippocampus as a spatial map: preliminary evidence from unit activity in the freely-moving rat","volume":"34","author":"O'Keefe","year":"1971","journal-title":"Brain Res"},{"key":"B71","article-title":"\u201cHow can we define intrinsic motivation?\u201d","author":"Oudeyer","year":"2008","journal-title":"Proceedings of the Eight International Conference on Epigenetic Robotics: Modeling Cognitive Development in Robotic Systems"},{"key":"B72","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2017.70","article-title":"Curiosity-driven exploration by self-supervised prediction","author":"Pathak","year":"2017","journal-title":"arXiv preprint arXiv:1705.05363"},{"key":"B73","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01832","article-title":"\u201cPoni: Potential functions for objectgoal navigation with interaction-free learning,\u201d","author":"Ramakrishnan","year":"2022","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B74","article-title":"Sayplan: grounding large language models using 3D scene graphs for scalable task planning","author":"Rana","year":"2023","journal-title":"arXiv preprint arXiv:2307.06135"},{"key":"B75","doi-asserted-by":"publisher","first-page":"28412","DOI":"10.1073\/pnas.2009821117","article-title":"A spiking neural program for sensorimotor control during foraging in flying insects","volume":"117","author":"Rapp","year":"2020","journal-title":"Proc. Nat. Acad. Sci"},{"key":"B76","first-page":"64","article-title":"A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and non-reinforcement","volume":"2","author":"Rescorla","year":"1972","journal-title":"Class. Condit. Curr. Res. Theory"},{"key":"B77","doi-asserted-by":"publisher","first-page":"814","DOI":"10.1126\/science.1255635","article-title":"Large environments reveal the statistical structure governing hippocampal representations","volume":"345","author":"Rich","year":"2014","journal-title":"Science"},{"key":"B78","doi-asserted-by":"crossref","first-page":"3437","DOI":"10.1109\/IROS55552.2023.10341922","article-title":"\u201cNerf-slam: real-time dense monocular slam with neural radiance fields,\u201d","volume-title":"2023 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)","author":"Rosinol","year":"2023"},{"key":"B79","doi-asserted-by":"publisher","first-page":"e1005768","DOI":"10.1371\/journal.pcbi.1005768","article-title":"Predictive representations can link model-based reinforcement learning to model-free mechanisms","volume":"13","author":"Russek","year":"2017","journal-title":"PLoS Comput. Biol"},{"key":"B80","doi-asserted-by":"publisher","first-page":"444","DOI":"10.1002\/cne.903340309","article-title":"Anatomy of the mushroom bodies in the honey bee brain: the neuronal connections of the alpha-lobe","volume":"334","author":"Rybak","year":"1993","journal-title":"J. Compar. Neurol"},{"key":"B81","article-title":"Episodic curiosity through reachability","author":"Savinov","year":"2019","journal-title":"arXiv preprint arXiv:1810.02274"},{"key":"B82","first-page":"52","article-title":"\u201cLearning long-horizon robot exploration strategies for multi-object search in continuous action spaces,\u201d","volume-title":"The International Symposium of Robotics Research","author":"Schmalstieg","year":"2022"},{"key":"B83","doi-asserted-by":"publisher","first-page":"8549","DOI":"10.1109\/LRA.2023.3329619","article-title":"Learning hierarchical interactive multi-object search for mobile manipulation","volume":"8","author":"Schmalstieg","year":"2023","journal-title":"IEEE Robot. Autom. Lett"},{"key":"B84","article-title":"Rapid exploration for open-world navigation with latent goal models","author":"Shah","year":"2023","journal-title":"arXiv preprint arXiv:2104.05859"},{"key":"B85","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2022.XVIII.019","article-title":"\u201cViKiNG: vision-based kilometer-scale navigation with geographic hints,\u201d","author":"Shah","year":"2022","journal-title":"Robotics: Science and Systems XVIII"},{"key":"B86","doi-asserted-by":"publisher","first-page":"e1500816","DOI":"10.1126\/science.1500816","article-title":"Connecting multiple spatial scales to decode the population activity of grid cells","volume":"1","author":"Stemmler","year":"2015","journal-title":"Sci. Adv"},{"key":"B87","doi-asserted-by":"publisher","first-page":"3069","DOI":"10.1016\/j.cub.2017.08.052","article-title":"An anatomically constrained model for path integration in the bee brain","volume":"27","author":"Stone","year":"2017","journal-title":"Curr. Biol"},{"key":"B88","doi-asserted-by":"publisher","first-page":"171785","DOI":"10.1098\/rsos.171785","article-title":"Multimodal integration and stimulus categorization in putative mushroom body output neurons of the honeybee","volume":"5","author":"Strube-Bloss","year":"2018","journal-title":"R. Soc. Open Sci"},{"key":"B89","doi-asserted-by":"publisher","first-page":"e54026","DOI":"10.7554\/eLife.54026","article-title":"A decentralised neural model explaining optimal integration of navigational strategies in insects","volume":"9","author":"Sun","year":"2020","journal-title":"Elife"},{"key":"B90","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1007\/BF00115009","article-title":"Learning to predict by the methods of temporal differences","volume":"3","author":"Sutton","year":"1988","journal-title":"Mach. Learn"},{"key":"B91","volume-title":"Reinforcement Learning: An Introduction. Adaptive Computation and Machine Learning Series","author":"Sutton","year":"2018"},{"key":"B92","article-title":"\u201cPolicy gradient methods for reinforcement learning with function approximation,\u201d","volume-title":"Advances in Neural Information Processing Systems","author":"Sutton","year":"1999"},{"key":"B93","article-title":"\u201cAttention is all you need,\u201d","volume-title":"Advances in neural information processing systems","author":"Vaswani","year":"2017"},{"key":"B94","first-page":"19","article-title":"\u201cContinual SLAM: beyond lifelong simultaneous localization and mapping through continual learning,\u201d","volume-title":"The International Symposium of Robotics Research","author":"V\u00f6disch","year":"2022"},{"key":"B95","doi-asserted-by":"publisher","DOI":"10.1101\/2023.12.20.572558","article-title":"High resolution outdoor videography of insects using fast lock-on tracking","author":"Vo-Doan","year":"2024","journal-title":"bioRxiv preprint, 2023.12.20.572558."},{"key":"B96","doi-asserted-by":"publisher","first-page":"e02395","DOI":"10.7554\/eLife.02395","article-title":"Shared mushroom body circuits underlie visual and olfactory memories in drosophila","volume":"3","author":"Vogt","year":"2014","journal-title":"Elife"},{"key":"B97","article-title":"MultiON: benchmarking semantic map memory using multi-object navigation","author":"Wani","year":"2020","journal-title":"arXiv preprint arXiv:2012.03912"},{"key":"B98","doi-asserted-by":"publisher","first-page":"152","DOI":"10.1016\/j.cognition.2017.05.020","article-title":"Wormholes in virtual space: from cognitive maps to cognitive graphs","volume":"166","author":"Warren","year":"2017","journal-title":"Cognition"},{"key":"B99","doi-asserted-by":"publisher","first-page":"jeb188094","DOI":"10.1242\/jeb.188094","article-title":"The internal maps of insects","volume":"222","author":"Webb","year":"2019","journal-title":"J. Exper. Biol"},{"key":"B100","doi-asserted-by":"publisher","first-page":"a053824","DOI":"10.1101\/lm.053824.123","article-title":"Beyond prediction error: 25 years of modeling the associations formed in the insect mushroom body","volume":"31","author":"Webb","year":"2024","journal-title":"Lear. Memory"},{"key":"B101","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1016\/j.cois.2016.02.011","article-title":"Neural mechanisms of insect navigation","volume":"15","author":"Webb","year":"2016","journal-title":"Curr. Opin. Insect Sci"},{"key":"B102","doi-asserted-by":"publisher","first-page":"e1012086","DOI":"10.1371\/journal.pcbi.1012086","article-title":"Learning with sparse reward in a gap junction network inspired by the insect mushroom body","volume":"20","author":"Wei","year":"2024","journal-title":"PLoS Comput. Biol"},{"key":"B103","article-title":"Hierarchical open-vocabulary 3D scene graphs for language-grounded robot navigation","author":"Werby","year":"2024","journal-title":"arXiv preprint arXiv:2403.17846"},{"key":"B104","doi-asserted-by":"publisher","first-page":"229","DOI":"10.1007\/BF00992696","article-title":"Simple statistical gradient-following algorithms for connectionist reinforcement learning","volume":"8","author":"Williams","year":"1992","journal-title":"Mach. Learn"},{"key":"B105","doi-asserted-by":"publisher","DOI":"10.1101\/2023.03.09.531867","article-title":"Neurons from pre-motor areas to the Mushroom bodies can orchestrate latent visual learning in navigating insects","author":"Wystrach","year":"2023","journal-title":"bioRxiv preprint 2023-03"},{"key":"B106","doi-asserted-by":"publisher","first-page":"1927","DOI":"10.1016\/j.cub.2020.02.082","article-title":"Rapid aversive and memory trace learning during route navigation in desert ants","volume":"30","author":"Wystrach","year":"2020","journal-title":"Curr. Biol"},{"key":"B107","doi-asserted-by":"publisher","first-page":"615","DOI":"10.1007\/s00359-014-0900-8","article-title":"Visual scanning behaviours and their role in the navigation of the Australian desert ant Melophorus bagoti","volume":"200","author":"Wystrach","year":"2014","journal-title":"J. Comp. Physiol. A Neuroethol. Sens. Neural Behav. Physiol"},{"key":"B108","doi-asserted-by":"publisher","first-page":"148","DOI":"10.1109\/MRA.2022.3213466","article-title":"Autonomous ground navigation in highly constrained spaces: lessons learned from the benchmark autonomous robot navigation challenge at icra 2022 [competitions]","volume":"29","author":"Xiao","year":"2022","journal-title":"IEEE Robot. Autom Mag"},{"key":"B109","doi-asserted-by":"publisher","first-page":"928","DOI":"10.1109\/LRA.2023.3234766","article-title":"Catch me if you hear me: audio-visual navigation in complex unmapped environments with moving sounds","volume":"8","author":"Younes","year":"2023","journal-title":"IEEE Robot. Autom. Lett"},{"key":"B110","doi-asserted-by":"publisher","first-page":"450","DOI":"10.1364\/JOSAA.20.000450","article-title":"Catchment areas of panoramic snapshots in outdoor scenes","volume":"20","author":"Zeil","year":"2003","journal-title":"J. Opt. Soc. Am. A"},{"key":"B111","doi-asserted-by":"publisher","first-page":"135426","DOI":"10.1109\/ACCESS.2020.3011438","article-title":"A survey on visual navigation for artificial agents with deep reinforcement learning","volume":"8","author":"Zeng","year":"2020","journal-title":"IEEE Access"},{"key":"B112","doi-asserted-by":"publisher","first-page":"2119","DOI":"10.1109\/TNNLS.2021.3105905","article-title":"Solving dynamic traveling salesman problems with deep reinforcement learning","volume":"34","author":"Zhang","year":"2023","journal-title":"IEEE Trans. Neural Netw. Lear. Syst"},{"key":"B113","doi-asserted-by":"publisher","first-page":"674","DOI":"10.26599\/TST.2021.9010012","article-title":"Deep reinforcement learning based mobile robot navigation: a review","volume":"26","author":"Zhu","year":"2021","journal-title":"Tsinghua Sci. Technol"},{"key":"B114","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01245","article-title":"\u201cNice-slam: neural implicit scalable encoding for slam. 2022 IEEE,\u201d","author":"Zhu","year":"2021","journal-title":"CVF Conference on Computer Vision and Pattern Recognition (CVPR)"}],"container-title":["Frontiers in Computational Neuroscience"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fncom.2024.1460006\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,9]],"date-time":"2024-09-09T04:53:35Z","timestamp":1725857615000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fncom.2024.1460006\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,9]]},"references-count":114,"alternative-id":["10.3389\/fncom.2024.1460006"],"URL":"https:\/\/doi.org\/10.3389\/fncom.2024.1460006","relation":{},"ISSN":["1662-5188"],"issn-type":[{"value":"1662-5188","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,9]]},"article-number":"1460006"}}