{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T14:28:22Z","timestamp":1740148102954,"version":"3.37.3"},"reference-count":18,"publisher":"Wiley","license":[{"start":{"date-parts":[[2018,1,1]],"date-time":"2018-01-01T00:00:00Z","timestamp":1514764800000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004735","name":"Natural Science Foundation of Hunan Province","doi-asserted-by":"publisher","award":["2017JJ3371"],"award-info":[{"award-number":["2017JJ3371"]}],"id":[{"id":"10.13039\/501100004735","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computational Intelligence and Neuroscience"],"published-print":{"date-parts":[[2018]]},"abstract":"<jats:p>Hierarchical reinforcement learning works on temporally extended actions or skills to facilitate learning. How to automatically form such abstraction is challenging, and many efforts tackle this issue in the options framework. While various approaches exist to construct options from different perspectives, few of them concentrate on options\u2019 adaptability during learning. This paper presents an algorithm to create options and enhance their quality online. Both aspects operate on detected communities of the learning environment\u2019s state transition graph. We first construct options from initial samples as the basis of online learning. Then a rule-based community revision algorithm is proposed to update graph partitions, based on which existing options can be continuously tuned. Experimental results in two problems indicate that options from initial samples may perform poorly in more complex environments, and our presented strategy can effectively improve options and get better results compared with flat reinforcement learning.<\/jats:p>","DOI":"10.1155\/2018\/2085721","type":"journal-article","created":{"date-parts":[[2018,4,23]],"date-time":"2018-04-23T19:31:02Z","timestamp":1524511862000},"page":"1-13","source":"Crossref","is-referenced-by-count":3,"title":["Constructing Temporally Extended Actions through Incremental Community Detection"],"prefix":"10.1155","volume":"2018","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9302-2230","authenticated-orcid":true,"given":"Xiao","family":"Xu","sequence":"first","affiliation":[{"name":"College of System Engineering, National University of Defense Technology, Changsha, Hunan 410073, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9008-4102","authenticated-orcid":true,"given":"Mei","family":"Yang","sequence":"additional","affiliation":[{"name":"College of System Engineering, National University of Defense Technology, Changsha, Hunan 410073, China"}]},{"given":"Ge","family":"Li","sequence":"additional","affiliation":[{"name":"College of System Engineering, National University of Defense Technology, Changsha, Hunan 410073, China"}]}],"member":"311","reference":[{"key":"2","doi-asserted-by":"publisher","DOI":"10.1016\/S0004-3702(99)00052-1"},{"key":"3","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1613\/jair.639","volume":"13","year":"2000","journal-title":"Journal of Artificial Intelligence Research"},{"key":"7","doi-asserted-by":"publisher","DOI":"10.1088\/1742-5468\/2008\/10\/P10008"},{"year":"1998","series-title":"Adaptive computation and machine learning","key":"8"},{"key":"16","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2013.04.010"},{"key":"19","doi-asserted-by":"publisher","DOI":"10.1007\/s12530-017-9193-9"},{"key":"20","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.76.036106"},{"key":"21","doi-asserted-by":"publisher","DOI":"10.1093\/acprof:oso\/9780199206650.001.0001"},{"key":"22","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.69.026113"},{"key":"24","doi-asserted-by":"publisher","DOI":"10.1016\/j.physrep.2009.11.002"},{"key":"25","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.80.056117"},{"key":"26","doi-asserted-by":"publisher","DOI":"10.1080\/0022250x.2001.9990249"},{"key":"28","doi-asserted-by":"publisher","DOI":"10.1038\/srep01825"},{"key":"29","doi-asserted-by":"publisher","DOI":"10.1023\/A:1022628806385"},{"volume-title":"A real-time detecting algorithm for tracking community structure of dynamic networks","year":"2012","key":"31"},{"volume":"abs\/1710","journal-title":"CoRR","year":"2017","key":"32"},{"key":"34","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2016.08.009"},{"key":"37","doi-asserted-by":"publisher","DOI":"10.1109\/TCIAIG.2017.2696045"}],"container-title":["Computational Intelligence and Neuroscience"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/downloads.hindawi.com\/journals\/cin\/2018\/2085721.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/cin\/2018\/2085721.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/cin\/2018\/2085721.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2019,10,16]],"date-time":"2019-10-16T18:41:05Z","timestamp":1571251265000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.hindawi.com\/journals\/cin\/2018\/2085721\/"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018]]},"references-count":18,"alternative-id":["2085721","2085721"],"URL":"https:\/\/doi.org\/10.1155\/2018\/2085721","relation":{},"ISSN":["1687-5265","1687-5273"],"issn-type":[{"type":"print","value":"1687-5265"},{"type":"electronic","value":"1687-5273"}],"subject":[],"published":{"date-parts":[[2018]]}}}