{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T06:43:48Z","timestamp":1740120228319,"version":"3.37.3"},"reference-count":26,"publisher":"World Scientific Pub Co Pte Ltd","issue":"10","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Int. J. Patt. Recogn. Artif. Intell."],"published-print":{"date-parts":[[2019,9]]},"abstract":"<jats:p> We proposed an improved variational Bayesian exploration-based active Sarsa (VBE-ASAR) algorithm, which tries to balance the exploration and exploitation dilemma, and speeds up the convergence rate. First, in the learning process, variational Bayesian method is adopted to measure the information gain, which is used as an exploration factor to construct an internal reward function for heuristic exploration. In addition, before the learning process, in order to improve the exploration performance, transfer learning is used to initialize the value function, where Bisimulation metric is introduced to measure the distance between two states from the source MDP and the target MDP, respectively. Finally, we apply the proposed algorithm to the cliff walking problem, and compare with the Sarsa algorithm, the Q-Learning algorithm, the VFT-Sarsa algorithm and the Bayesian Sarsa (BS) algorithm. Experimental results show that the VBE-ASAR algorithm has a faster learning rate. <\/jats:p>","DOI":"10.1142\/s0218001419510054","type":"journal-article","created":{"date-parts":[[2019,1,11]],"date-time":"2019-01-11T04:22:27Z","timestamp":1547180547000},"page":"1951005","source":"Crossref","is-referenced-by-count":1,"title":["Variational Bayesian Exploration-Based Active Sarsa Algorithm"],"prefix":"10.1142","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8720-9071","authenticated-orcid":false,"given":"Qiming","family":"Fu","sequence":"first","affiliation":[{"name":"Institute of Electronics and Information Engineering, Suzhou University of Science and Technology, Suzhou, Jiangsu 215009, P. R. China"},{"name":"Jiangsu Key Laboratory of Intelligent Building Energy Efficiency, Suzhou University of Science and Technology, Suzhou, Jiangsu 215009, P. R. China"},{"name":"Suzhou Key Laboratory of Mobile Networking and Applied Technologies, Suzhou University of Science and Technology, Suzhou, Jiangsu 215009, P. R. China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhengxia","family":"Yang","sequence":"additional","affiliation":[{"name":"Institute of Electronics and Information Engineering, Suzhou University of Science and Technology, Suzhou, Jiangsu 215009, P. R. China"},{"name":"Jiangsu Key Laboratory of Intelligent Building Energy Efficiency, Suzhou University of Science and Technology, Suzhou, Jiangsu 215009, P. R. China"},{"name":"Suzhou Key Laboratory of Mobile Networking and Applied Technologies, Suzhou University of Science and Technology, Suzhou, Jiangsu 215009, P. R. China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"You","family":"Lu","sequence":"additional","affiliation":[{"name":"Institute of Electronics and Information Engineering, Suzhou University of Science and Technology, Suzhou, Jiangsu 215009, P. R. China"},{"name":"Jiangsu Key Laboratory of Intelligent Building Energy Efficiency, Suzhou University of Science and Technology, Suzhou, Jiangsu 215009, P. R. China"},{"name":"Suzhou Key Laboratory of Mobile Networking and Applied Technologies, Suzhou University of Science and Technology, Suzhou, Jiangsu 215009, P. R. China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hongjie","family":"Wu","sequence":"additional","affiliation":[{"name":"Institute of Electronics and Information Engineering, Suzhou University of Science and Technology, Suzhou, Jiangsu 215009, P. R. China"},{"name":"Jiangsu Key Laboratory of Intelligent Building Energy Efficiency, Suzhou University of Science and Technology, Suzhou, Jiangsu 215009, P. R. China"},{"name":"Suzhou Key Laboratory of Mobile Networking and Applied Technologies, Suzhou University of Science and Technology, Suzhou, Jiangsu 215009, P. R. China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fuyuan","family":"Hu","sequence":"additional","affiliation":[{"name":"Institute of Electronics and Information Engineering, Suzhou University of Science and Technology, Suzhou, Jiangsu 215009, P. R. China"},{"name":"Jiangsu Key Laboratory of Intelligent Building Energy Efficiency, Suzhou University of Science and Technology, Suzhou, Jiangsu 215009, P. R. China"},{"name":"Suzhou Key Laboratory of Mobile Networking and Applied Technologies, Suzhou University of Science and Technology, Suzhou, Jiangsu 215009, P. R. China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jianping","family":"Chen","sequence":"additional","affiliation":[{"name":"Institute of Electronics and Information Engineering, Suzhou University of Science and Technology, Suzhou, Jiangsu 215009, P. R. China"},{"name":"Jiangsu Key Laboratory of Intelligent Building Energy Efficiency, Suzhou University of Science and Technology, Suzhou, Jiangsu 215009, P. R. China"},{"name":"Suzhou Key Laboratory of Mobile Networking and Applied Technologies, Suzhou University of Science and Technology, Suzhou, Jiangsu 215009, P. R. China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"219","published-online":{"date-parts":[[2019,9,9]]},"reference":[{"issue":"1","key":"S0218001419510054BIB001","first-page":"1","volume":"45","author":"Abed-Alguni B. H.","year":"2017","journal-title":"Arab. J. Sci. Eng."},{"key":"S0218001419510054BIB002","first-page":"1703","volume-title":"Int. Conf. Information Knowledge Management","author":"Athukorala K.","year":"2015"},{"key":"S0218001419510054BIB003","doi-asserted-by":"publisher","DOI":"10.19153\/cleiej.21.2.1"},{"volume-title":"Variational Algorithms for Approximate Bayesian Inference","year":"2003","author":"Beal M. J.","key":"S0218001419510054BIB004"},{"key":"S0218001419510054BIB005","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN.2010.5596815"},{"key":"S0218001419510054BIB006","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2013.2283574"},{"key":"S0218001419510054BIB007","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-017-17237-w"},{"key":"S0218001419510054BIB008","first-page":"104","volume-title":"The 20th Conf. Uncertainty in Artificial Intelligence","author":"Ferns N.","year":"2004"},{"issue":"11","key":"S0218001419510054BIB009","first-page":"2157","volume":"42","author":"Fu Q. M.","year":"2014","journal-title":"J. Electron."},{"key":"S0218001419510054BIB010","doi-asserted-by":"publisher","DOI":"10.1016\/S0004-3702(02)00376-4"},{"issue":"1","key":"S0218001419510054BIB011","first-page":"507","volume":"29","author":"Notsu A.","year":"2017","journal-title":"J. Japan Soc. Fuzzy Theo. Intell. Inf."},{"key":"S0218001419510054BIB012","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2014.2340191"},{"key":"S0218001419510054BIB013","first-page":"1105","volume-title":"Int. Conf. Computer Science","author":"Seldin Y.","year":"2011"},{"key":"S0218001419510054BIB014","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2017.7952670"},{"key":"S0218001419510054BIB015","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-22887-2_5"},{"volume-title":"Reinforcement Learning: An Introduction","year":"1998","author":"Sutton R. S.","key":"S0218001419510054BIB016"},{"key":"S0218001419510054BIB017","first-page":"22","volume-title":"Int. Conf. Neural Information Processing Systems","author":"Tang H.","year":"2017"},{"key":"S0218001419510054BIB018","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2012.09.110"},{"key":"S0218001419510054BIB019","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2014.2327636"},{"key":"S0218001419510054BIB020","first-page":"737","volume-title":"Int. Conf. Computational Intelligence","author":"Tijsma A. D.","year":"2017"},{"key":"S0218001419510054BIB021","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2016.09.141"},{"key":"S0218001419510054BIB022","doi-asserted-by":"publisher","DOI":"10.1109\/ICISCE.2017.104"},{"volume-title":"Int. Conf. Robotics and Automation","year":"2017","author":"Xie C.","key":"S0218001419510054BIB023"},{"key":"S0218001419510054BIB024","first-page":"425","volume-title":"Int. Conf. Cloud Computing and Big Data Analysis","author":"Xu Z. X.","year":"2017"},{"key":"S0218001419510054BIB025","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-26532-2_13"},{"key":"S0218001419510054BIB026","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2014.11.018"}],"container-title":["International Journal of Pattern Recognition and Artificial Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S0218001419510054","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2019,9,9]],"date-time":"2019-09-09T22:21:50Z","timestamp":1568067710000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/abs\/10.1142\/S0218001419510054"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,9]]},"references-count":26,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2019,9,9]]},"published-print":{"date-parts":[[2019,9]]}},"alternative-id":["10.1142\/S0218001419510054"],"URL":"https:\/\/doi.org\/10.1142\/s0218001419510054","relation":{},"ISSN":["0218-0014","1793-6381"],"issn-type":[{"type":"print","value":"0218-0014"},{"type":"electronic","value":"1793-6381"}],"subject":[],"published":{"date-parts":[[2019,9]]}}}