{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,6]],"date-time":"2024-08-06T15:40:05Z","timestamp":1722958805762},"reference-count":80,"publisher":"World Scientific Pub Co Pte Ltd","issue":"09","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Int. J. Neur. Syst."],"published-print":{"date-parts":[[2020,9]]},"abstract":"<jats:p>Hippocampal place cells and interneurons in mammals have stable place fields and theta phase precession profiles that encode spatial environmental information. Hippocampal CA1 neurons can represent the animal\u2019s location and prospective information about the goal location. Reinforcement learning (RL) algorithms such as Q-learning have been used to build the navigation models. However, the traditional Q-learning ([Formula: see text]Q-learning) limits the reward function once the animals arrive at the goal location, leading to unsatisfactory location accuracy and convergence rates. Therefore, we proposed a revised version of the Q-learning algorithm, dynamical Q-learning ([Formula: see text]Q-learning), which assigns the reward function adaptively to improve the decoding performance. Firing rate was the input of the neural network of [Formula: see text]Q-learning and was used to predict the movement direction. On the other hand, phase precession was the input of the reward function to update the weights of [Formula: see text]Q-learning. Trajectory predictions using [Formula: see text]Q- and [Formula: see text]Q-learning were compared by the root mean squared error (RMSE) between the actual and predicted rat trajectories. Using [Formula: see text]Q-learning, significantly higher prediction accuracy and faster convergence rate were obtained compared with [Formula: see text]Q-learning in all cell types. Moreover, combining place cells and interneurons with theta phase precession improved the convergence rate and prediction accuracy. The proposed [Formula: see text]Q-learning algorithm is a quick and more accurate method to perform trajectory reconstruction and prediction.<\/jats:p>","DOI":"10.1142\/s0129065720500483","type":"journal-article","created":{"date-parts":[[2020,5,28]],"date-time":"2020-05-28T10:17:32Z","timestamp":1590661052000},"page":"2050048","source":"Crossref","is-referenced-by-count":1,"title":["Enhancement of Hippocampal Spatial Decoding Using a Dynamic Q-Learning Method With a Relative Reward Using Theta Phase Precession"],"prefix":"10.1142","volume":"30","author":[{"given":"Bo-Wei","family":"Chen","sequence":"first","affiliation":[{"name":"Department of Biomedical Engineering, National Yang Ming University, No. 155, Section 2, Linong Street, Taipei 11221, Taiwan"},{"name":"Department of Mechanical Engineering, National Cheng Kung University, No. 1 University Road, Tainan 70101, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shih-Hung","family":"Yang","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, National Cheng Kung University, No. 1 University Road, Tainan 70101, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yu-Chun","family":"Lo","sequence":"additional","affiliation":[{"name":"The Ph.D. Program for Neural Regenerative Medicine, College of Medical Science and Technology, Taipei Medical University, No. 250 Wu-Xing Street, Taipei 11031, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ching-Fu","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Biomedical Engineering, National Yang Ming University, No. 155, Section 2, Linong Street, Taipei 11221, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Han-Lin","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Biomedical Engineering, National Yang Ming University, No. 155, Section 2, Linong Street, Taipei 11221, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chen-Yang","family":"Hsu","sequence":"additional","affiliation":[{"name":"Department of Biomedical Engineering, National Yang Ming University, No. 155, Section 2, Linong Street, Taipei 11221, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yun-Ting","family":"Kuo","sequence":"additional","affiliation":[{"name":"Department of Biomedical Engineering, National Yang Ming University, No. 155, Section 2, Linong Street, Taipei 11221, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jung-Chen","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Biomedical Engineering, National Yang Ming University, No. 155, Section 2, Linong Street, Taipei 11221, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sheng-Huang","family":"Lin","sequence":"additional","affiliation":[{"name":"Department of Neurology, Hualien Tzu Chi Hospital, Buddhist Tzu Chi Medical Foundation, No. 707, Section 3, Chung Yang Road, Hualien 97002, Taiwan"},{"name":"Department of Neurology, School of Medicine, Tzu Chi University, No. 701, Section 3, Zhongyang Road, Hualien 97004, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Han-Chi","family":"Pan","sequence":"additional","affiliation":[{"name":"National Laboratory Animal Center, No. 99, Lane 130, Section 1, Academia Road, Taipei 11571, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sheng-Wei","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, National Cheng Kung University, No. 1 University Road, Tainan 70101, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiao","family":"Yu","sequence":"additional","affiliation":[{"name":"Department of Neurology of the Second Affiliated Hospital, Interdisciplinary Institute of Neuroscience and Technology, Key Laboratory of Medical Neurobiology of Zhejiang Province, Zhejiang University School of Medicine, Hangzhou 310029, P. R. China"},{"name":"College of Biomedical Engineering and Instrument Science, Key Laboratory for Biomedical Engineering of Ministry of Education, Zhejiang University, Hangzhou 310027, P. R. China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Boyi","family":"Qu","sequence":"additional","affiliation":[{"name":"Department of Neurology of the Second Affiliated Hospital, Interdisciplinary Institute of Neuroscience and Technology, Key Laboratory of Medical Neurobiology of Zhejiang Province, Zhejiang University School of Medicine, Hangzhou 310029, P. R. China"},{"name":"College of Biomedical Engineering and Instrument Science, Key Laboratory for Biomedical Engineering of Ministry of Education, Zhejiang University, Hangzhou 310027, P. R. China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chao-Hung","family":"Kuo","sequence":"additional","affiliation":[{"name":"Department of Biomedical Engineering, National Yang Ming University, No. 155, Section 2, Linong Street, Taipei 11221, Taiwan"},{"name":"Department of Neurosurgery, Neurological Institute, Taipei Veterans General Hospital, No. 201, Section 2, Shipai Road, Taipei 11217, Taiwan"},{"name":"Department of Neurological Surgery, University of Washington, No. 1959 NE Pacific Street, Seattle, WA 98195-6470, U.S.A."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"You-Yin","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Biomedical Engineering, National Yang Ming University, No. 155, Section 2, Linong Street, Taipei 11221, Taiwan"},{"name":"The Ph.D. Program for Neural Regenerative Medicine, College of Medical Science and Technology, Taipei Medical University, No. 250 Wu-Xing Street, Taipei 11031, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hsin-Yi","family":"Lai","sequence":"additional","affiliation":[{"name":"Department of Neurology of the Second Affiliated Hospital, Interdisciplinary Institute of Neuroscience and Technology, Key Laboratory of Medical Neurobiology of Zhejiang Province, Zhejiang University School of Medicine, Hangzhou 310029, P. R. China"},{"name":"College of Biomedical Engineering and Instrument Science, Key Laboratory for Biomedical Engineering of Ministry of Education, Zhejiang University, Hangzhou 310027, P. R. China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"219","published-online":{"date-parts":[[2020,8,12]]},"reference":[{"key":"S0129065720500483BIB001","doi-asserted-by":"publisher","DOI":"10.1098\/rstb.2012.0510"},{"key":"S0129065720500483BIB002","doi-asserted-by":"publisher","DOI":"10.1038\/nature14396"},{"key":"S0129065720500483BIB003","doi-asserted-by":"publisher","DOI":"10.1142\/S0129065707001093"},{"key":"S0129065720500483BIB004","first-page":"87","volume":"6","author":"Burgess N.","year":"1995","journal-title":"Int. J. Neural Syst."},{"key":"S0129065720500483BIB005","first-page":"95","volume":"6","author":"Knierim J. J.","year":"1995","journal-title":"Int. J. Neural Syst."},{"key":"S0129065720500483BIB006","doi-asserted-by":"publisher","DOI":"10.1016\/0014-4886(73)90290-2"},{"key":"S0129065720500483BIB007","doi-asserted-by":"publisher","DOI":"10.1016\/0014-4886(75)90213-7"},{"key":"S0129065720500483BIB008","doi-asserted-by":"publisher","DOI":"10.1016\/0006-8993(71)90358-1"},{"key":"S0129065720500483BIB009","doi-asserted-by":"publisher","DOI":"10.1016\/0014-4886(76)90055-8"},{"key":"S0129065720500483BIB010","doi-asserted-by":"publisher","DOI":"10.1038\/381425a0"},{"key":"S0129065720500483BIB011","doi-asserted-by":"publisher","DOI":"10.1098\/rstb.2012.0532"},{"key":"S0129065720500483BIB012","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.2882-06.2006"},{"key":"S0129065720500483BIB013","doi-asserted-by":"publisher","DOI":"10.1002\/hipo.450030307"},{"key":"S0129065720500483BIB014","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1098-1063(1996)6:2<149::AID-HIPO6>3.0.CO;2-K"},{"key":"S0129065720500483BIB015","doi-asserted-by":"publisher","DOI":"10.1038\/nn.2106"},{"key":"S0129065720500483BIB016","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-017-08609-3"},{"key":"S0129065720500483BIB017","doi-asserted-by":"publisher","DOI":"10.1038\/nature08499"},{"key":"S0129065720500483BIB018","doi-asserted-by":"publisher","DOI":"10.1152\/jn.2000.83.5.2602"},{"key":"S0129065720500483BIB019","doi-asserted-by":"publisher","DOI":"10.1002\/hipo.20509"},{"key":"S0129065720500483BIB020","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2013.06.022"},{"key":"S0129065720500483BIB021","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.18-18-07411.1998"},{"key":"S0129065720500483BIB022","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2011.00085"},{"key":"S0129065720500483BIB023","doi-asserted-by":"publisher","DOI":"10.1038\/nature12112"},{"key":"S0129065720500483BIB024","doi-asserted-by":"publisher","DOI":"10.1152\/jn.1998.79.2.1017"},{"key":"S0129065720500483BIB025","doi-asserted-by":"publisher","DOI":"10.1109\/TNSRE.2005.847368"},{"key":"S0129065720500483BIB026","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.14-12-07235.1994"},{"key":"S0129065720500483BIB027","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.5393-12.2014"},{"key":"S0129065720500483BIB028","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.95.6.3182"},{"key":"S0129065720500483BIB029","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-019-10139-7"},{"key":"S0129065720500483BIB030","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.09-08-02764.1989"},{"key":"S0129065720500483BIB031","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.09-08-02737.1989"},{"key":"S0129065720500483BIB032","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.15986"},{"key":"S0129065720500483BIB033","doi-asserted-by":"publisher","DOI":"10.1038\/nn.2648"},{"key":"S0129065720500483BIB034","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1098-1063(2000)10:1<1::AID-HIPO1>3.0.CO;2-1"},{"key":"S0129065720500483BIB035","doi-asserted-by":"publisher","DOI":"10.1007\/s10827-008-0090-x"},{"key":"S0129065720500483BIB036","doi-asserted-by":"publisher","DOI":"10.1385\/NI:3:3:197"},{"key":"S0129065720500483BIB037","doi-asserted-by":"publisher","DOI":"10.1007\/BF00992698"},{"volume-title":"Reinforcement Learning: An Introduction","year":"2018","author":"Sutton R. S.","key":"S0129065720500483BIB038"},{"key":"S0129065720500483BIB039","doi-asserted-by":"publisher","DOI":"10.3233\/ICA-160531"},{"key":"S0129065720500483BIB040","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2016.08.155"},{"key":"S0129065720500483BIB041","doi-asserted-by":"publisher","DOI":"10.1002\/int.20105"},{"key":"S0129065720500483BIB042","doi-asserted-by":"publisher","DOI":"10.1113\/jphysiol.2010.194274"},{"key":"S0129065720500483BIB043","doi-asserted-by":"publisher","DOI":"10.1152\/jn.01170.2003"},{"key":"S0129065720500483BIB044","doi-asserted-by":"publisher","DOI":"10.1152\/jn.01200.2006"},{"key":"S0129065720500483BIB045","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-21344-1_16"},{"key":"S0129065720500483BIB046","doi-asserted-by":"publisher","DOI":"10.3389\/fncir.2015.00087"},{"key":"S0129065720500483BIB047","doi-asserted-by":"publisher","DOI":"10.1523\/ENEURO.0423-17.2019"},{"key":"S0129065720500483BIB048","doi-asserted-by":"publisher","DOI":"10.1007\/s11517-009-0494-4"},{"key":"S0129065720500483BIB049","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2560\/8\/3\/036003"},{"key":"S0129065720500483BIB050","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.0276-15.2015"},{"key":"S0129065720500483BIB051","doi-asserted-by":"publisher","DOI":"10.1016\/S0896-6273(00)80525-5"},{"key":"S0129065720500483BIB052","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1098-1063(1996)6:4<347::AID-HIPO1>3.0.CO;2-I"},{"key":"S0129065720500483BIB053","doi-asserted-by":"publisher","DOI":"10.1016\/j.jneumeth.2012.03.007"},{"key":"S0129065720500483BIB054","doi-asserted-by":"publisher","DOI":"10.1002\/hipo.20253"},{"key":"S0129065720500483BIB055","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0172395"},{"key":"S0129065720500483BIB056","volume-title":"Neuro-dynamic Programming","volume":"5","author":"Bertsekas D. P.","year":"1996"},{"key":"S0129065720500483BIB057","doi-asserted-by":"publisher","DOI":"10.1109\/TBME.2019.2906640"},{"key":"S0129065720500483BIB058","volume-title":"Handbook of Stochastic Methods","volume":"3","author":"Gardiner C. W.","year":"1985"},{"issue":"1","key":"S0129065720500483BIB059","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1111\/j.2517-6161.1977.tb01600.x","volume":"39","author":"Dempster A. P.","year":"1977","journal-title":"J. R. Stat. Soc.: Ser. B"},{"key":"S0129065720500483BIB060","doi-asserted-by":"publisher","DOI":"10.1162\/neco.2009.10-08-893"},{"key":"S0129065720500483BIB061","doi-asserted-by":"publisher","DOI":"10.1113\/jphysiol.2004.078915"},{"key":"S0129065720500483BIB062","doi-asserted-by":"publisher","DOI":"10.1002\/hipo.22659"},{"key":"S0129065720500483BIB063","doi-asserted-by":"publisher","DOI":"10.1038\/nn.4486"},{"key":"S0129065720500483BIB064","doi-asserted-by":"publisher","DOI":"10.1002\/hipo.22456"},{"key":"S0129065720500483BIB065","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2006.07.004"},{"key":"S0129065720500483BIB066","doi-asserted-by":"publisher","DOI":"10.1016\/j.beproc.2016.12.012"},{"key":"S0129065720500483BIB067","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2018.08.004"},{"key":"S0129065720500483BIB068","doi-asserted-by":"publisher","DOI":"10.1038\/nn.3909"},{"key":"S0129065720500483BIB069","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.0682-16.2016"},{"key":"S0129065720500483BIB070","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.2595-15.2016"},{"key":"S0129065720500483BIB071","doi-asserted-by":"publisher","DOI":"10.1038\/nn.3138"},{"key":"S0129065720500483BIB072","doi-asserted-by":"publisher","DOI":"10.1162\/089976603765202622"},{"key":"S0129065720500483BIB073","doi-asserted-by":"publisher","DOI":"10.1002\/hipo.20115"},{"key":"S0129065720500483BIB074","doi-asserted-by":"publisher","DOI":"10.1016\/j.tics.2010.09.004"},{"key":"S0129065720500483BIB075","doi-asserted-by":"publisher","DOI":"10.1002\/hipo.20501"},{"key":"S0129065720500483BIB076","doi-asserted-by":"publisher","DOI":"10.1038\/nrn1932"},{"key":"S0129065720500483BIB077","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.10094"},{"key":"S0129065720500483BIB078","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuron.2015.07.006"},{"key":"S0129065720500483BIB079","doi-asserted-by":"publisher","DOI":"10.1080\/02664769922160"},{"key":"S0129065720500483BIB080","doi-asserted-by":"publisher","DOI":"10.1016\/S0378-5955(01)00325-2"}],"container-title":["International Journal of Neural Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S0129065720500483","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,6]],"date-time":"2024-08-06T15:26:00Z","timestamp":1722957960000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/abs\/10.1142\/S0129065720500483"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,12]]},"references-count":80,"journal-issue":{"issue":"09","published-print":{"date-parts":[[2020,9]]}},"alternative-id":["10.1142\/S0129065720500483"],"URL":"https:\/\/doi.org\/10.1142\/s0129065720500483","relation":{},"ISSN":["0129-0657","1793-6462"],"issn-type":[{"type":"print","value":"0129-0657"},{"type":"electronic","value":"1793-6462"}],"subject":[],"published":{"date-parts":[[2020,8,12]]}}}