{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,10]],"date-time":"2024-08-10T07:10:32Z","timestamp":1723273832389},"publisher-location":"California","reference-count":0,"publisher":"International Joint Conferences on Artificial Intelligence Organization","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2019,8]]},"abstract":"<jats:p>We consider the Markov Decision Process (MDP) of selecting a subset of items at each step, termed the Select-MDP (S-MDP). The large state and action spaces of S-MDPs make them intractable to solve with typical reinforcement learning (RL) algorithms especially when the number of items is huge. In this paper, we present a deep RL algorithm to solve this issue by adopting the following key ideas. First, we convert the original S-MDP into an Iterative Select-MDP (IS-MDP), which is equivalent to the S-MDP in terms of optimal actions. IS-MDP decomposes a joint action of selecting K items simultaneously into K iterative selections resulting in the decrease of actions at the expense of an exponential increase of states. Second, we overcome this state space explosion by exploiting a special symmetry in IS-MDPs with novel weight shared Q-networks, which provably maintain sufficient expressive power. Various experiments demonstrate that our approach works well even when the item space is large and that it scales to environments with item spaces different from those used in training.<\/jats:p>","DOI":"10.24963\/ijcai.2019\/481","type":"proceedings-article","created":{"date-parts":[[2019,7,28]],"date-time":"2019-07-28T07:46:05Z","timestamp":1564299965000},"page":"3467-3474","source":"Crossref","is-referenced-by-count":4,"title":["Solving Continual Combinatorial Selection via Deep Reinforcement Learning"],"prefix":"10.24963","author":[{"given":"Hyungseok","family":"Song","sequence":"first","affiliation":[{"name":"School of Electrical Engineering, KAIST, Daejeon, South Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hyeryung","family":"Jang","sequence":"additional","affiliation":[{"name":"Informatics, King\u2019s College London, London, United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hai H.","family":"Tran","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering, KAIST, Daejeon, South Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Se-eun","family":"Yoon","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering, KAIST, Daejeon, South Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kyunghwan","family":"Son","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering, KAIST, Daejeon, South Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Donggyu","family":"Yun","sequence":"additional","affiliation":[{"name":"Naver Corporation, Seongnam, South Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hyoju","family":"Chung","sequence":"additional","affiliation":[{"name":"Naver Corporation, Seongnam, South Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yung","family":"Yi","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering, KAIST, Daejeon, South Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"10584","event":{"number":"28","sponsor":["International Joint Conferences on Artificial Intelligence Organization (IJCAI)"],"acronym":"IJCAI-2019","name":"Twenty-Eighth International Joint Conference on Artificial Intelligence {IJCAI-19}","start":{"date-parts":[[2019,8,10]]},"theme":"Artificial Intelligence","location":"Macao, China","end":{"date-parts":[[2019,8,16]]}},"container-title":["Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence"],"original-title":[],"deposited":{"date-parts":[[2019,7,28]],"date-time":"2019-07-28T07:49:37Z","timestamp":1564300177000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ijcai.org\/proceedings\/2019\/481"}},"subtitle":[],"proceedings-subject":"Artificial Intelligence Research Articles","short-title":[],"issued":{"date-parts":[[2019,8]]},"references-count":0,"URL":"https:\/\/doi.org\/10.24963\/ijcai.2019\/481","relation":{},"subject":[],"published":{"date-parts":[[2019,8]]}}}