{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T14:09:15Z","timestamp":1772806155598,"version":"3.50.1"},"reference-count":106,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,2,9]],"date-time":"2023-02-09T00:00:00Z","timestamp":1675900800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Comput. Neurosci."],"abstract":"<jats:p>Recent experimental observations have shown that the reactivation of hippocampal place cells (PC) during sleep or wakeful immobility depicts trajectories that can go around barriers and can flexibly adapt to a changing maze layout. However, existing computational models of replay fall short of generating such layout-conforming replay, restricting their usage to simple environments, like linear tracks or open fields. In this paper, we propose a computational model that generates layout-conforming replay and explains how such replay drives the learning of flexible navigation in a maze. First, we propose a Hebbian-like rule to learn the inter-PC synaptic strength during exploration. Then we use a continuous attractor network (CAN) with feedback inhibition to model the interaction among place cells and hippocampal interneurons. The activity bump of place cells drifts along paths in the maze, which models layout-conforming replay. During replay in sleep, the synaptic strengths from place cells to striatal medium spiny neurons (MSN) are learned by a novel dopamine-modulated three-factor rule to store place-reward associations. During goal-directed navigation, the CAN periodically generates replay trajectories from the animal's location for path planning, and the trajectory leading to a maximal MSN activity is followed by the animal. We have implemented our model into a high-fidelity virtual rat in the MuJoCo physics simulator. Extensive experiments have demonstrated that its superior flexibility during navigation in a maze is due to a continuous re-learning of inter-PC and PC-MSN synaptic strength.<\/jats:p>","DOI":"10.3389\/fncom.2023.1053097","type":"journal-article","created":{"date-parts":[[2023,2,9]],"date-time":"2023-02-09T01:19:08Z","timestamp":1675905548000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["A computational model of learning flexible navigation in a maze by layout-conforming replay of place cells"],"prefix":"10.3389","volume":"17","author":[{"given":"Yuanxiang","family":"Gao","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2023,2,9]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"128","DOI":"10.1016\/0166-2236(88)90137-3","article-title":"Is contiguity detection in classical conditioning a system or a cellular property? learning in aplysia suggests a possible molecular site","volume":"11","author":"Abrams","year":"1988","journal-title":"Trends Neurosci"},{"key":"B2","doi-asserted-by":"publisher","first-page":"18428","DOI":"10.1073\/pnas.1421056111","article-title":"Place cells in the hippocampus: eleven maps for eleven rooms","volume":"111","author":"Alme","year":"2014","journal-title":"Proc. Natl. Acad. Sci. U.S.A"},{"key":"B3","doi-asserted-by":"publisher","first-page":"1696","DOI":"10.1111\/j.1460-9568.2011.07653.x","article-title":"Local remapping of place cell firing in the Tolman detour task","volume":"33","author":"Alvernhe","year":"2011","journal-title":"Eur. J. Neurosci"},{"key":"B4","doi-asserted-by":"publisher","first-page":"1145","DOI":"10.1016\/j.neuron.2014.04.021","article-title":"Neurons in the ventral striatum exhibit cell-type-specific representations of outcome during learning","volume":"82","author":"Atallah","year":"2014","journal-title":"Neuron"},{"key":"B5","doi-asserted-by":"publisher","first-page":"161","DOI":"10.3389\/fncom.2013.00161","article-title":"A computational model for preplay in the hippocampus","volume":"7","author":"Azizi","year":"2013","journal-title":"Front. Comput. Neurosci"},{"key":"B6","doi-asserted-by":"publisher","first-page":"429","DOI":"10.1038\/s41586-018-0102-6","article-title":"Vector-based navigation using grid-like representations in artificial agents","volume":"557","author":"Banino","year":"2018","journal-title":"Nature"},{"key":"B7","doi-asserted-by":"publisher","first-page":"7738","DOI":"10.1103\/PhysRevE.58.7738","article-title":"Attractor neural networks storing multiple space representations: a model for hippocampal place fields","volume":"58","author":"Battaglia","year":"1998","journal-title":"Phys. Rev"},{"key":"B8","first-page":"152","article-title":"Simulation of a thalamocortical circuit for computing directional heading in the rat","volume":"8","author":"Blair","year":"1996","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B9","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1162\/neco.1996.8.1.85","article-title":"A model of spatial map formation in the hippocampus of the rat","volume":"8","author":"Blum","year":"1996","journal-title":"Neural Comput"},{"key":"B10","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1002\/hipo.450050304","article-title":"Simulation of spatial learning in the Morris water maze by a neural network model of the hippocampal formation and nucleus accumbens","volume":"5","author":"Brown","year":"1995","journal-title":"Hippocampus"},{"key":"B11","doi-asserted-by":"publisher","first-page":"563","DOI":"10.1016\/j.neuron.2019.05.041","article-title":"Neuromodulation of spike-timing-dependent plasticity: past, present, and future","volume":"103","author":"Brzosko","year":"2019","journal-title":"Neuron Rev"},{"key":"B12","doi-asserted-by":"publisher","first-page":"749","DOI":"10.1002\/(SICI)1098-1063(1996)6:6andlt;749::AID-HIPO16andgt;3.0.CO;2-0","article-title":"Neuronal computations underlying the firing of place cells and their role in navigation","volume":"6","author":"Burgess","year":"1996","journal-title":"Hippocampus"},{"key":"B13","doi-asserted-by":"publisher","first-page":"1073","DOI":"10.1002\/hipo.22488","article-title":"Hippocampal sharp wave-ripple: a cognitive biomarker for episodic memory and planning","volume":"25","author":"Buzsaki","year":"2015","journal-title":"Hippocampus"},{"key":"B14","volume-title":"Introduction to Algorithms, 3rd Edn","author":"Cormen","year":"2009"},{"key":"B15","doi-asserted-by":"publisher","first-page":"483","DOI":"10.1113\/jphysiol.2003.050120","article-title":"Ca2+-activated K+ (BK) channel inactivation contributes to spike broadening during repetitive firing in the rat lateral amygdala","volume":"552","author":"Faber","year":"2003","journal-title":"J. Physiol"},{"key":"B16","doi-asserted-by":"publisher","first-page":"2851","DOI":"10.1523\/JNEUROSCI.21-08-02851.2001","article-title":"Modulation of hippocampal and amygdalar-evoked activity of nucleus accumbens neurons by dopamine: cellular mechanisms of input selection","volume":"21","author":"Floresco","year":"2001","journal-title":"J. Neurosci"},{"key":"B17","doi-asserted-by":"publisher","first-page":"1679","DOI":"10.1523\/JNEUROSCI.1407-19.2020","article-title":"Continuous representations of speed by striatal medium spiny neurons","volume":"40","author":"Fobbs","year":"2020","journal-title":"J. Neurosci"},{"key":"B18","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1002\/(SICI)1098-1063(2000)10:1andlt;1::AID-HIPO1andgt;3.0.CO;2-1","article-title":"A model of hippocampally dependent navigation, using the temporal difference learning rule","volume":"10","author":"Foster","year":"2000","journal-title":"Hippocampus"},{"key":"B19","doi-asserted-by":"publisher","first-page":"680","DOI":"10.1038\/nature04587","article-title":"Reverse replay of behavioural sequences in hippocampal place cells during the awake state","volume":"440","author":"Foster","year":"2006","journal-title":"Nature"},{"key":"B20","doi-asserted-by":"publisher","first-page":"18531","DOI":"10.1523\/JNEUROSCI.1278-13.2013","article-title":"Control of basal ganglia output by direct and indirect pathway projection neurons","volume":"33","author":"Freeze","year":"2013","journal-title":"J. Neurosci"},{"key":"B21","article-title":"\u201cAddressing function approximation error in actor-critic methods,\u201d","volume-title":"International Conference on Machine Learning (ICML)","author":"Fujimoto","year":"2018"},{"key":"B22","doi-asserted-by":"publisher","first-page":"752","DOI":"10.1162\/neco.2009.07-08-824","article-title":"A moving bump in a continuous manifold: a comprehensive study of the tracking dynamics of continuous attractor neural networks","volume":"22","author":"Fung","year":"2010","journal-title":"Neural Comput"},{"key":"B23","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1016\/j.neuron.2018.06.008","article-title":"A dedicated population for reward coding in the hippocampus","volume":"99","author":"Gauthier","year":"2018","journal-title":"Neuron"},{"key":"B24","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1023\/A:1008820728122","article-title":"Learning navigational maps through potentiation and modulation of hippocampal place cells","volume":"4","author":"Gerstner","year":"1997","journal-title":"J. Comput. Neurosci"},{"key":"B25","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3389\/fncir.2018.00053","article-title":"Eligibility traces and plasticity on behavioral time scales: experimental support of neoHebbian three-factor learning rules","volume":"12","author":"Gerstner","year":"2018","journal-title":"Front. Neural Circ"},{"key":"B26","doi-asserted-by":"publisher","first-page":"84","DOI":"10.3389\/fncom.2017.00084","article-title":"Predictive place-cell sequences for goal-finding emerge from goal memory and the cognitive map: a computational model","volume":"11","author":"Gonner","year":"2017","journal-title":"Front. Comput. Neurosci"},{"key":"B27","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pcbi.1002235","article-title":"Grid cells, place cells, and geodesic generalization for spatial reinforcement learning","volume":"7","author":"Gustafson","year":"2011","journal-title":"PLoS Comput. Biol"},{"key":"B28","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1016\/j.neuron.2017.06.011","article-title":"Neuroscience-inspired artificial intelligence","volume":"95","author":"Hassabis","year":"2017","journal-title":"Neuron"},{"key":"B29","volume-title":"Neural Networks: A Comprehensive Foundation","author":"Haykin","year":"1998"},{"key":"B30","doi-asserted-by":"publisher","first-page":"1069","DOI":"10.1016\/S0006-3495(96)79653-4","article-title":"Ca2+ buffering and action potential-evoked Ca2+ signaling in dendrites of pyramidal neurons","volume":"70","author":"Helmchen","year":"1996","journal-title":"Biophys. J"},{"key":"B31","doi-asserted-by":"publisher","first-page":"472","DOI":"10.1523\/JNEUROSCI.2864-06.2007","article-title":"Goal-related activity in hippocampal place cells","volume":"27","author":"Hok","year":"2007","journal-title":"J. Neurosci"},{"key":"B32","doi-asserted-by":"publisher","first-page":"1648","DOI":"10.1073\/pnas.0913991107","article-title":"Neurodynamics of mental exploration","volume":"107","author":"Hopfield","year":"2009","journal-title":"Proc. Natl. Acad. Sci"},{"key":"B33","doi-asserted-by":"publisher","first-page":"575","DOI":"10.1038\/nature12475","article-title":"Prolonged dopamine signalling in striatum signals proximity and value of distant rewards","volume":"500","author":"Howe","year":"2013","journal-title":"Nature"},{"key":"B34","doi-asserted-by":"publisher","first-page":"6840","DOI":"10.1523\/JNEUROSCI.0731-09.2009","article-title":"Hippocampal place cell firing patterns can induce long-term synaptic plasticity in vitro","volume":"29","author":"Isaac","year":"2009","journal-title":"J. Neurosci"},{"key":"B35","doi-asserted-by":"publisher","first-page":"2828","DOI":"10.1523\/JNEUROSCI.3773-10.2011","article-title":"Cell assembly sequences arising from spike threshold adaptation keep track of time in the hippocampus","volume":"31","author":"Itskov","year":"2011","journal-title":"J. Neurosci"},{"key":"B36","doi-asserted-by":"publisher","first-page":"2443","DOI":"10.1093\/cercor\/bhl152","article-title":"Solving the distal reward problem through linkage of STDP and dopamine signaling","volume":"10","author":"Izhikevich","year":"2007","journal-title":"Cereb. Cortex"},{"key":"B37","doi-asserted-by":"publisher","first-page":"1454","DOI":"10.1126\/science.1217230","article-title":"Awake hippocampal sharp-wave ripples support spatial memory","volume":"336","author":"Jadhav","year":"2012","journal-title":"Science"},{"key":"B38","doi-asserted-by":"publisher","first-page":"375","DOI":"10.1016\/S0301-0082(03)00085-6","article-title":"Dopamine: a potential substrate for synaptic plasticity and memory mechanisms","volume":"69","author":"Jay","year":"2003","journal-title":"Progr. Neurobiol"},{"key":"B39","doi-asserted-by":"publisher","first-page":"1163","DOI":"10.1016\/j.neunet.2005.08.009","article-title":"Hippocampal replay contributes to within session learning in a temporal difference reinforcement learning model","volume":"18","author":"Johnson","year":"2005","journal-title":"Neural Netw"},{"key":"B40","doi-asserted-by":"publisher","first-page":"12176","DOI":"10.1523\/JNEUROSCI.3761-07.2007","article-title":"Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point","volume":"27","author":"Johnson","year":"2007","journal-title":"J. Neurosci"},{"key":"B41","doi-asserted-by":"publisher","first-page":"407","DOI":"10.1038\/s41583-021-00467-3","article-title":"Spine dynamics in the brain, mental disorders and artificial neural networks","volume":"22","author":"Kasai","year":"2021","journal-title":"Nat. Rev. Neurosci"},{"key":"B42","doi-asserted-by":"publisher","first-page":"247","DOI":"10.1016\/j.neuron.2015.08.037","article-title":"Dopamine prediction errors in reward learning and addiction: from theory to neural circuitry","volume":"88","author":"Keiflin","year":"2015","journal-title":"Neuron"},{"key":"B43","doi-asserted-by":"publisher","first-page":"4498","DOI":"10.1103\/PhysRevE.59.4498","article-title":"Hebbian learning and spiking neurons","volume":"59","author":"Kempter","year":"1999","journal-title":"Phys. Rev. E"},{"key":"B44","doi-asserted-by":"publisher","first-page":"1600","DOI":"10.1016\/j.cell.2020.11.013","article-title":"A unified framework for dopamine signals across timescales","volume":"183","author":"Kim","year":"2020","journal-title":"Cell"},{"key":"B45","doi-asserted-by":"publisher","first-page":"3481","DOI":"10.1111\/ejn.12722","article-title":"Striatal firing rate reflects head movement velocity","volume":"40","author":"Kim","year":"2014","journal-title":"Eur. J. Neurosci"},{"key":"B46","doi-asserted-by":"publisher","first-page":"622","DOI":"10.1038\/nature09159","article-title":"Regulation of parkinsonian motor behaviours by optogenetic control of basal ganglia circuitry","volume":"466","author":"Kravitz","year":"2010","journal-title":"Nature"},{"key":"B47","doi-asserted-by":"publisher","first-page":"e1000173","DOI":"10.1371\/journal.pbio.1000173","article-title":"Hippocampus leads ventral striatum in replay of place-reward information","volume":"7","author":"Lansink","year":"2009","journal-title":"PLoS Biol"},{"key":"B48","doi-asserted-by":"publisher","first-page":"1183","DOI":"10.1016\/S0896-6273(02)01096-6","article-title":"Memory of sequential experience in the hippocampus during slow wave sleep","volume":"36","author":"Lee","year":"2002","journal-title":"Neuron"},{"key":"B49","doi-asserted-by":"publisher","first-page":"346","DOI":"10.1109\/TEC.1961.5219222","article-title":"An algorithm for path connections and its applications","volume":"10","author":"Lee","year":"1961","journal-title":"IRE Trans. Electron. Comput"},{"key":"B50","doi-asserted-by":"publisher","first-page":"3547","DOI":"10.1523\/JNEUROSCI.2693-17.2018","article-title":"Coordinated ramping of dorsal striatal pathways preceding food approach and consumption","volume":"38","author":"London","year":"2018","journal-title":"J. Neurosci"},{"key":"B51","doi-asserted-by":"publisher","first-page":"353","DOI":"10.1126\/science.1204622","article-title":"Linking context with reward: a functional circuit from hippocampal CA3 to ventral tegmental area","volume":"333","author":"Luo","year":"2011","journal-title":"Science"},{"key":"B52","doi-asserted-by":"publisher","first-page":"663","DOI":"10.1038\/nrn1932","article-title":"Path integration and the neural basis of the \u2018cognitive map'","volume":"7","author":"McNaughton","year":"2006","journal-title":"Nature Reviews Neuroscience"},{"key":"B53","article-title":"\u201cDeep neuroethology of a virtual rodent,\u201d","volume-title":"International Conference on Learning Representations (ICLR)","author":"Merel","year":"2020"},{"key":"B54","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41467-019-13239-6","article-title":"Hierarchical motor control in mammals and machines","volume":"10","author":"Merel","year":"2019","journal-title":"Nat. Commun"},{"key":"B55","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/ncomms11552","article-title":"Symmetric spike timing-dependent plasticity at CA3-CA3 synapses optimizes storage and recall in autoassociative networks","volume":"7","author":"Mishra","year":"2016","journal-title":"Nat. Commun"},{"key":"B56","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1016\/j.cobeha.2020.02.017","article-title":"Learning structures: Predictive representations, replay, and generalization","volume":"32","author":"Momennejad","year":"2020","journal-title":"Curr. Opin. Behav. Sci"},{"key":"B57","doi-asserted-by":"publisher","first-page":"457","DOI":"10.1016\/j.tins.2012.04.009","article-title":"Reinforcement learning: computing the temporal difference of values via distinct corticostriatal pathways","volume":"35","author":"Morita","year":"2012","journal-title":"Trends Neurosci"},{"key":"B58","doi-asserted-by":"publisher","first-page":"243","DOI":"10.1002\/hipo.450010306","article-title":"The hippocampus as a cognitive graph (abridged version)","volume":"1","author":"Muller","year":"1991","journal-title":"Hippocampus"},{"key":"B59","doi-asserted-by":"publisher","first-page":"663","DOI":"10.1085\/jgp.107.6.663","article-title":"The hippocampus as a cognitive graph","volume":"107","author":"Muller","year":"1996","journal-title":"J. Gen. Physiol"},{"key":"B60","doi-asserted-by":"crossref","DOI":"10.1090\/mmono\/047","volume-title":"Stochastic Approximation and Recursive Estimation","author":"Nevelson","year":"1976"},{"key":"B61","doi-asserted-by":"publisher","first-page":"425","DOI":"10.1038\/381425a0","article-title":"Geometrical determinants of the place fields of hippocampal neurons","volume":"381","author":"O'Keefe","year":"1996","journal-title":"Nature"},{"key":"B62","doi-asserted-by":"publisher","first-page":"2011","DOI":"10.1523\/JNEUROSCI.1451-21.2021","article-title":"A conditioned place preference for heroin is signaled by increased dopamine and direct pathway activity and decreased indirect pathway activity in the nucleus accumbens","volume":"42","author":"O'Neal","year":"2022","journal-title":"J. Neurosci"},{"key":"B63","doi-asserted-by":"publisher","first-page":"1619","DOI":"10.1152\/physrev.00007.2017","article-title":"Hippocampal GABAergic inhibitory interneurons","volume":"97","author":"Pelkey","year":"2017","journal-title":"Physiol. Rev"},{"key":"B64","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1002\/hipo.22824","article-title":"The content of hippocampal \u201creplay\u201d","volume":"30","author":"Pfeiffer","year":"2017","journal-title":"Hippocampus"},{"key":"B65","doi-asserted-by":"publisher","first-page":"74","DOI":"10.1038\/nature12112","article-title":"Hippocampal place-cell sequences depict future paths to remembered goals","volume":"497","author":"Pfeiffer","year":"2013","journal-title":"Nature"},{"key":"B66","doi-asserted-by":"publisher","first-page":"98","DOI":"10.3389\/fncom.2013.00098","article-title":"Rapid, parallel path planning by propagating wavefronts of spiking neural activity","volume":"7","author":"Ponulak","year":"2013","journal-title":"Front. Comput. Neurosci"},{"key":"B67","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1016\/j.cobeha.2017.06.003","article-title":"Remembering goal locations","volume":"17","author":"Poucet","year":"2007","journal-title":"Curr. Opin. Behav. Sci"},{"key":"B68","doi-asserted-by":"publisher","first-page":"94","DOI":"10.1002\/hipo.22355","article-title":"Short-term plasticity based network model of place cells dynamics","volume":"25","author":"Romani","year":"2015","journal-title":"Hippocampus"},{"key":"B69","doi-asserted-by":"publisher","first-page":"1","DOI":"10.7554\/eLife.66175","article-title":"Mice in a labyrinth show rapid learning, sudden insight, and efficient exploration","volume":"10","author":"Rosenberg","year":"2021","journal-title":"Elife"},{"key":"B70","doi-asserted-by":"publisher","first-page":"e1005768","DOI":"10.1371\/journal.pcbi.1005768","article-title":"Predictive representations can link model-based reinforcement learning to model-free mechanisms","volume":"13","author":"Russek","year":"2017","journal-title":"PLoS Comput. Biol"},{"key":"B71","doi-asserted-by":"publisher","first-page":"5900","DOI":"10.1523\/JNEUROSCI.17-15-05900.1997","article-title":"Path integration and cognitive mapping in a continuous attractor neural network model","volume":"17","author":"Samsonovich","year":"1997","journal-title":"J. Neurosci"},{"key":"B72","doi-asserted-by":"publisher","first-page":"11385","DOI":"10.1523\/JNEUROSCI.0867-14.2014","article-title":"Mechanisms of sharp wave initiation and ripple generation","volume":"34","author":"Schlingloff","year":"2014","journal-title":"J. Neurosci"},{"key":"B73","article-title":"\u201cHigh-dimensional continuous control using generalized advantage estimation,\u201d","volume-title":"International Conference on Learning Representations (ICLR)","author":"Schulman","year":"2016"},{"key":"B74","doi-asserted-by":"publisher","first-page":"1593","DOI":"10.1126\/science.275.5306.1593","article-title":"A neural substrate of prediction and reward","volume":"275","author":"Schultz","year":"1997","journal-title":"Science"},{"key":"B75","article-title":"\u201cTrue online TD(\u03bb),\u201d","volume-title":"International Conference on Machine Learning (ICML)","author":"Seijen","year":"2014"},{"key":"B76","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1111\/j.1469-7793.1999.00135.x","article-title":"The role of BK-type Ca2+-dependent K+ channels in spike broadening during repetitive firing in rat hippocampal pyramidal cells","volume":"521","author":"Shao","year":"1999","journal-title":"J. Physiol"},{"key":"B77","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1007\/BF00114726","article-title":"Reinforcement learning with replacing eligibility traces","volume":"22","author":"Singh","year":"1996","journal-title":"Mach. Learn"},{"key":"B78","doi-asserted-by":"publisher","first-page":"926","DOI":"10.1016\/j.neuron.2018.04.015","article-title":"Cocaine place conditioning strengthens location-specific hippocampal coupling to the nucleus accumbens","volume":"98","author":"Sjulson","year":"2018","journal-title":"Neuron"},{"key":"B79","first-page":"173","article-title":"A model of the neural basis of the rat's sense of direction","volume":"7","author":"Skaggs","year":"1995","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B80","doi-asserted-by":"publisher","first-page":"8455","DOI":"10.1523\/JNEUROSCI.18-20-08455.1998","article-title":"Spatial firing properties of hippocampal CA1 populations in an environment containing two visually identical regions","volume":"18","author":"Skaggs","year":"1998","journal-title":"J. Neurosci"},{"key":"B81","doi-asserted-by":"publisher","first-page":"725","DOI":"10.1016\/j.neuron.2019.11.022","article-title":"Dorsal and ventral hippocampal sharp-wave ripples activate distinct nucleus accumbens networks","volume":"105","author":"Sosa","year":"2020","journal-title":"Neuron"},{"key":"B82","doi-asserted-by":"publisher","first-page":"1","DOI":"10.7554\/eLife.69499","article-title":"Continuous attractors for dynamic memories","volume":"10","author":"Spalla","year":"2021","journal-title":"Elife"},{"key":"B83","doi-asserted-by":"publisher","first-page":"1643","DOI":"10.1038\/nn.4650","article-title":"The hippocampus as a predictive map","volume":"20","author":"Stachenfeld","year":"2017","journal-title":"Nat. Neurosci"},{"key":"B84","doi-asserted-by":"publisher","first-page":"467","DOI":"10.1016\/j.neuron.2014.06.023","article-title":"Pyramidal cell-interneuron interactions underlie hippocampal ripple oscillations","volume":"83","author":"Stark","year":"2014","journal-title":"Neuron"},{"key":"B85","doi-asserted-by":"publisher","first-page":"450","DOI":"10.1016\/j.neuron.2019.01.052","article-title":"Hippocampal reactivation of random trajectories resembling Brownian diffusion","volume":"102","author":"Stella","year":"2019","journal-title":"Neuron"},{"key":"B86","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1016\/S0893-6080(03)00210-7","article-title":"Self-organising continuous attractor networks with multiple activity packets, and the representation of space","volume":"17","author":"Stringer","year":"2004","journal-title":"Neural Netw"},{"key":"B87","doi-asserted-by":"publisher","first-page":"429","DOI":"10.1088\/0954-898X_13_4_301","article-title":"Self-organizing continuous attractor networks and path integration: two-dimensional models of place cells","volume":"13","author":"Stringer","year":"2002","journal-title":"Netw. Comput. Neural Syst"},{"key":"B88","doi-asserted-by":"crossref","DOI":"10.1016\/B978-1-55860-141-3.50030-4","article-title":"\u201cIntegrated architectures for learning, planning, and reacting based on approximating dynamic programming,\u201d","volume-title":"International Conference on Machine Learning (ICML)","author":"Sutton","year":"1990"},{"key":"B89","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton","year":"1998"},{"key":"B90","volume-title":"Reinforcement Learning: An Introduction, 2nd Edn","author":"Sutton","year":"2018"},{"key":"B91","article-title":"\u201cPolicy gradient methods for reinforcement learning with function approximation,\u201d","volume-title":"Neural Information Processing Systems (NeurIPS)","author":"Sutton","year":"1999"},{"key":"B92","article-title":"\u201cMujoco: a physics engine for model-based contro,\u201d","volume-title":"IEEE\/RSJ International Conference on Intelligent Robots and Systems","author":"Todorov","year":"2012"},{"key":"B93","doi-asserted-by":"publisher","first-page":"189","DOI":"10.1037\/h0061626","article-title":"Cognitive maps in rats and men","volume":"55","author":"Tolman","year":"1948","journal-title":"Psychol. Rev"},{"key":"B94","first-page":"215","article-title":"\u201cInsight\u201d in rats","volume":"4","author":"Tolman","year":"1930","journal-title":"Univer. California Publicat. Psychol"},{"key":"B95","doi-asserted-by":"publisher","first-page":"1393","DOI":"10.1016\/j.cell.2018.12.037","article-title":"A hippocampus-accumbens tripartite neuronal motif guides appetitive memory in space","volume":"176","author":"Trouche","year":"2019","journal-title":"Cell"},{"key":"B96","doi-asserted-by":"publisher","first-page":"481","DOI":"10.1002\/(SICI)1098-1063(1999)9:4andlt;481::AID-HIPO14andgt;3.0.CO;2-S","article-title":"Attractor neural network models of spatial maps in hippocampus","volume":"9","author":"Tsodyks","year":"1999","journal-title":"Hippocampus"},{"key":"B97","doi-asserted-by":"publisher","first-page":"2843","DOI":"10.1523\/JNEUROSCI.4869-10.2011","article-title":"Theta phase precession in rat ventral striatum links place and reward information","volume":"31","author":"van der Meer","year":"2011","journal-title":"J. Neurosci"},{"key":"B98","doi-asserted-by":"publisher","first-page":"1549","DOI":"10.1152\/jn.1998.79.3.1549","article-title":"Calcium coding and adaptive temporal computation in cortical pyramidal neurons","volume":"79","author":"Wang","year":"1998","journal-title":"J. Neurophysiol"},{"key":"B99","doi-asserted-by":"publisher","first-page":"373","DOI":"10.1146\/annurev-neuro-072116-031109","article-title":"Neural circuitry of reward prediction error","volume":"40","author":"Watabe-Uchida","year":"2017","journal-title":"Annu. Rev. Neurosci"},{"key":"B100","doi-asserted-by":"publisher","first-page":"1547","DOI":"10.1016\/j.neuron.2022.02.002","article-title":"Flexible rerouting of hippocampal replay sequences around changing barriers in the absence of global place field remapping","volume":"110","author":"Widloski","year":"2022","journal-title":"Neuron"},{"key":"B101","doi-asserted-by":"publisher","first-page":"229","DOI":"10.1007\/BF00992696","article-title":"Simple statistical gradient-following algorithms for connectionist reinforcement learning","volume":"8","author":"Williams","year":"1992","journal-title":"Mach. Learn"},{"key":"B102","doi-asserted-by":"publisher","first-page":"3177","DOI":"10.1016\/j.cub.2019.07.089","article-title":"Locomotor and hippocampal processing converge in the lateral septum","volume":"19","author":"Wirtshafter","year":"2019","journal-title":"Curr. Biol"},{"key":"B103","doi-asserted-by":"publisher","first-page":"994","DOI":"10.1162\/neco.2008.10-06-378","article-title":"Dynamics and computation of continuous attractors","volume":"20","author":"Wu","year":"2008","journal-title":"Neural Comput"},{"key":"B104","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1016\/j.neuron.2018.11.015","article-title":"Assembly responses of hippocampal CA1 place cells predict learned behavior in goal-directed spatial tasks on the radial eight-arm maze","volume":"101","author":"Xu","year":"2019","journal-title":"Neuron"},{"key":"B105","doi-asserted-by":"publisher","first-page":"1616","DOI":"10.1126\/science.1255514","article-title":"A critical time window for dopamine actions on the structural plasticity of dendritic spines","volume":"345","author":"Yagishita","year":"2014","journal-title":"Science"},{"key":"B106","doi-asserted-by":"publisher","first-page":"2112","DOI":"10.1523\/JNEUROSCI.16-06-02112.1996","article-title":"Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: a theory","volume":"16","author":"Zhang","year":"1996","journal-title":"J. Neurosci"}],"container-title":["Frontiers in Computational Neuroscience"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fncom.2023.1053097\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,9]],"date-time":"2023-02-09T01:19:24Z","timestamp":1675905564000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fncom.2023.1053097\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,9]]},"references-count":106,"alternative-id":["10.3389\/fncom.2023.1053097"],"URL":"https:\/\/doi.org\/10.3389\/fncom.2023.1053097","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2022.09.16.508350","asserted-by":"object"}]},"ISSN":["1662-5188"],"issn-type":[{"value":"1662-5188","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,9]]},"article-number":"1053097"}}