{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,27]],"date-time":"2026-01-27T11:11:55Z","timestamp":1769512315720,"version":"3.49.0"},"publisher-location":"California","reference-count":0,"publisher":"International Joint Conferences on Artificial Intelligence Organization","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2019,8]]},"abstract":"<jats:p>Learning rational behaviors in First-person-shooter (FPS) games is a challenging task for Reinforcement Learning (RL) with the primary difficulties of huge action space and insufficient exploration. To address this, we propose a hierarchical agent based on combined options with intrinsic rewards to drive exploration. Specifically, we present a hierarchical model that works in a manager-worker fashion over two levels of hierarchy. The high-level manager learns a policy over options, and the low-level workers, motivated by intrinsic reward, learn to execute the options. Performance is further improved with environmental signals appropriately harnessed. Extensive experiments demonstrate that our trained bot significantly outperforms the alternative RL-based models on FPS games requiring maze solving and combat skills, etc.\nNotably, we achieved first place in VDAIC 2018 Track(1).<\/jats:p>","DOI":"10.24963\/ijcai.2019\/482","type":"proceedings-article","created":{"date-parts":[[2019,7,28]],"date-time":"2019-07-28T07:46:05Z","timestamp":1564299965000},"page":"3475-3482","source":"Crossref","is-referenced-by-count":21,"title":["Playing FPS Games With Environment-Aware Hierarchical Reinforcement Learning"],"prefix":"10.24963","author":[{"given":"Shihong","family":"Song","sequence":"first","affiliation":[{"name":"Institute for AI, Tsinghua University"},{"name":"Department of Computer Science and Technology, Tsinghua University"},{"name":"Beijing National Research Center for Information Science and Technology"},{"name":"Tsinghua Laboratory of Brain and Intelligence Lab"},{"name":"Center for Intelligent Connected Vehicles and Transportation, Tsinghua University"}]},{"given":"Jiayi","family":"Weng","sequence":"additional","affiliation":[{"name":"Institute for AI, Tsinghua University"},{"name":"Department of Computer Science and Technology, Tsinghua University"},{"name":"Beijing National Research Center for Information Science and Technology"},{"name":"Tsinghua Laboratory of Brain and Intelligence Lab"},{"name":"Center for Intelligent Connected Vehicles and Transportation, Tsinghua University"}]},{"given":"Hang","family":"Su","sequence":"additional","affiliation":[{"name":"Institute for AI, Tsinghua University"},{"name":"Department of Computer Science and Technology, Tsinghua University"},{"name":"Beijing National Research Center for Information Science and Technology"},{"name":"Tsinghua Laboratory of Brain and Intelligence Lab"},{"name":"Center for Intelligent Connected Vehicles and Transportation, Tsinghua University"}]},{"given":"Dong","family":"Yan","sequence":"additional","affiliation":[{"name":"Institute for AI, Tsinghua University"},{"name":"Department of Computer Science and Technology, Tsinghua University"},{"name":"Beijing National Research Center for Information Science and Technology"},{"name":"Tsinghua Laboratory of Brain and Intelligence Lab"},{"name":"Center for Intelligent Connected Vehicles and Transportation, Tsinghua University"}]},{"given":"Haosheng","family":"Zou","sequence":"additional","affiliation":[{"name":"Institute for AI, Tsinghua University"},{"name":"Department of Computer Science and Technology, Tsinghua University"},{"name":"Beijing National Research Center for Information Science and Technology"},{"name":"Tsinghua Laboratory of Brain and Intelligence Lab"},{"name":"Center for Intelligent Connected Vehicles and Transportation, Tsinghua University"}]},{"given":"Jun","family":"Zhu","sequence":"additional","affiliation":[{"name":"Institute for AI, Tsinghua University"},{"name":"Department of Computer Science and Technology, Tsinghua University"},{"name":"Beijing National Research Center for Information Science and Technology"},{"name":"Tsinghua Laboratory of Brain and Intelligence Lab"},{"name":"Center for Intelligent Connected Vehicles and Transportation, Tsinghua University"}]}],"member":"10584","event":{"name":"Twenty-Eighth International Joint Conference on Artificial Intelligence {IJCAI-19}","theme":"Artificial Intelligence","location":"Macao, China","acronym":"IJCAI-2019","number":"28","sponsor":["International Joint Conferences on Artificial Intelligence Organization (IJCAI)"],"start":{"date-parts":[[2019,8,10]]},"end":{"date-parts":[[2019,8,16]]}},"container-title":["Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence"],"original-title":[],"deposited":{"date-parts":[[2019,7,28]],"date-time":"2019-07-28T07:49:37Z","timestamp":1564300177000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ijcai.org\/proceedings\/2019\/482"}},"subtitle":[],"proceedings-subject":"Artificial Intelligence Research Articles","short-title":[],"issued":{"date-parts":[[2019,8]]},"references-count":0,"URL":"https:\/\/doi.org\/10.24963\/ijcai.2019\/482","relation":{},"subject":[],"published":{"date-parts":[[2019,8]]}}}