{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T14:28:17Z","timestamp":1740148097374,"version":"3.37.3"},"reference-count":20,"publisher":"Wiley","license":[{"start":{"date-parts":[[2016,1,1]],"date-time":"2016-01-01T00:00:00Z","timestamp":1451606400000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"crossref","award":["ZYGX2014J052","613153","11261015","61300192"],"award-info":[{"award-number":["ZYGX2014J052","613153","11261015","61300192"]}],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100004761","name":"Natural Science Foundation of Hainan Province","doi-asserted-by":"publisher","award":["ZYGX2014J052","613153","11261015","61300192"],"award-info":[{"award-number":["ZYGX2014J052","613153","11261015","61300192"]}],"id":[{"id":"10.13039\/501100004761","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["ZYGX2014J052","613153","11261015","61300192"],"award-info":[{"award-number":["ZYGX2014J052","613153","11261015","61300192"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["ZYGX2014J052","613153","11261015","61300192"],"award-info":[{"award-number":["ZYGX2014J052","613153","11261015","61300192"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computational Intelligence and Neuroscience"],"published-print":{"date-parts":[[2016]]},"abstract":"<jats:p>By combining with sparse kernel methods, least-squares temporal difference (LSTD) algorithms can construct the feature dictionary automatically and obtain a better generalization ability. However, the previous kernel-based LSTD algorithms do not consider regularization and their sparsification processes are batch or offline, which hinder their widespread applications in online learning problems. In this paper, we combine the following five techniques and propose two novel kernel recursive LSTD algorithms: (i) online sparsification, which can cope with unknown state regions and be used for online learning, (ii)<mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\" id=\"M1\"><mml:mrow><mml:msub><mml:mrow><mml:mi>L<\/mml:mi><\/mml:mrow><mml:mrow><mml:mn mathvariant=\"normal\">2<\/mml:mn><\/mml:mrow><\/mml:msub><\/mml:mrow><\/mml:math>and<mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\" id=\"M2\"><mml:mrow><mml:msub><mml:mrow><mml:mi>L<\/mml:mi><\/mml:mrow><mml:mrow><mml:mn mathvariant=\"normal\">1<\/mml:mn><\/mml:mrow><\/mml:msub><\/mml:mrow><\/mml:math>regularization, which can avoid overfitting and eliminate the influence of noise, (iii) recursive least squares, which can eliminate matrix-inversion operations and reduce computational complexity, (iv) a sliding-window approach, which can avoid caching all history samples and reduce the computational cost, and (v) the fixed-point subiteration and online pruning, which can make<mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\" id=\"M3\"><mml:mrow><mml:msub><mml:mrow><mml:mi>L<\/mml:mi><\/mml:mrow><mml:mrow><mml:mn mathvariant=\"normal\">1<\/mml:mn><\/mml:mrow><\/mml:msub><\/mml:mrow><\/mml:math>regularization easy to implement. Finally, simulation results on two 50-state chain problems demonstrate the effectiveness of our algorithms.<\/jats:p>","DOI":"10.1155\/2016\/2305854","type":"journal-article","created":{"date-parts":[[2016,6,29]],"date-time":"2016-06-29T23:23:14Z","timestamp":1467242594000},"page":"1-11","source":"Crossref","is-referenced-by-count":1,"title":["Kernel Recursive Least-Squares Temporal Difference Algorithms with Sparsification and Regularization"],"prefix":"10.1155","volume":"2016","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8297-9803","authenticated-orcid":true,"given":"Chunyuan","family":"Zhang","sequence":"first","affiliation":[{"name":"School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China"},{"name":"College of Information Science and Technology, Hainan University, Haikou 570228, China"}]},{"given":"Qingxin","family":"Zhu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China"}]},{"given":"Xinzheng","family":"Niu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China"}]}],"member":"311","reference":[{"issue":"1\u20133","key":"1","first-page":"33","volume":"22","year":"1996","journal-title":"Machine Learning"},{"key":"2","doi-asserted-by":"publisher","DOI":"10.1023\/a:1017936530646"},{"key":"3","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1613\/jair.946","volume":"16","year":"2002","journal-title":"Journal of Artificial Intelligence Research"},{"key":"4","doi-asserted-by":"publisher","DOI":"10.1016\/j.sigpro.2015.04.024"},{"key":"6","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2013.02.010"},{"year":"2004","key":"8"},{"year":"2010","key":"9"},{"key":"10","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2011.2178446"},{"key":"11","doi-asserted-by":"publisher","DOI":"10.1109\/TSP.2004.830985"},{"key":"12","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2013.2258936"},{"key":"16","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-23808-6_1"},{"key":"17","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2013.2270561"},{"key":"18","doi-asserted-by":"publisher","DOI":"10.1155\/2015\/481375"},{"key":"19","doi-asserted-by":"publisher","DOI":"10.1007\/11881070_8"},{"key":"20","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2007.899161"},{"year":"1998","key":"23"},{"year":"2002","key":"24"},{"year":"2006","key":"25"},{"key":"27","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1991.3.2.213"},{"key":"31","doi-asserted-by":"publisher","DOI":"10.1162\/1532443041827907"}],"container-title":["Computational Intelligence and Neuroscience"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/downloads.hindawi.com\/journals\/cin\/2016\/2305854.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/cin\/2016\/2305854.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/cin\/2016\/2305854.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,5,17]],"date-time":"2020-05-17T00:28:12Z","timestamp":1589675292000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.hindawi.com\/journals\/cin\/2016\/2305854\/"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016]]},"references-count":20,"alternative-id":["2305854","2305854"],"URL":"https:\/\/doi.org\/10.1155\/2016\/2305854","relation":{},"ISSN":["1687-5265","1687-5273"],"issn-type":[{"type":"print","value":"1687-5265"},{"type":"electronic","value":"1687-5273"}],"subject":[],"published":{"date-parts":[[2016]]}}}