{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,7]],"date-time":"2024-08-07T07:35:44Z","timestamp":1723016144924},"publisher-location":"California","reference-count":0,"publisher":"International Joint Conferences on Artificial Intelligence Organization","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,7]]},"abstract":"<jats:p>Stochastic variance-reduced gradient (SVRG) is an optimization method originally designed for tackling machine learning problems with a finite sum structure. SVRG was later shown to work for policy evaluation, a problem in reinforcement learning in which one aims to estimate the value function of a given policy. SVRG makes use of gradient estimates at two scales. At the slower scale, SVRG computes a full gradient over the whole dataset, which could lead to prohibitive computation costs. In this work, we show that two variants of SVRG for policy evaluation could significantly diminish the number of gradient calculations while preserving a linear convergence speed. More importantly, our theoretical result implies that one does not need to use the entire dataset in every epoch of SVRG when it is applied to policy evaluation with linear function approximation. Our experiments demonstrate large computational savings provided by the proposed methods.<\/jats:p>","DOI":"10.24963\/ijcai.2020\/374","type":"proceedings-article","created":{"date-parts":[[2020,7,8]],"date-time":"2020-07-08T12:12:10Z","timestamp":1594210330000},"page":"2697-2703","source":"Crossref","is-referenced-by-count":0,"title":["SVRG for Policy Evaluation with Fewer Gradient Evaluations"],"prefix":"10.24963","author":[{"given":"Zilun","family":"Peng","sequence":"first","affiliation":[{"name":"Mila, McGill University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ahmed","family":"Touati","sequence":"additional","affiliation":[{"name":"Mila, Universit\u00e9 de Montr\u00e9al"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Pascal","family":"Vincent","sequence":"additional","affiliation":[{"name":"Mila, Universit\u00e9 de Montr\u00e9al"},{"name":"Facebook AI Research"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Doina","family":"Precup","sequence":"additional","affiliation":[{"name":"Mila, McGill University"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"10584","event":{"number":"28","sponsor":["International Joint Conferences on Artificial Intelligence Organization (IJCAI)"],"acronym":"IJCAI-PRICAI-2020","name":"Twenty-Ninth International Joint Conference on Artificial Intelligence and Seventeenth Pacific Rim International Conference on Artificial Intelligence {IJCAI-PRICAI-20}","start":{"date-parts":[[2020,7,11]]},"theme":"Artificial Intelligence","location":"Yokohama, Japan","end":{"date-parts":[[2020,7,17]]}},"container-title":["Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence"],"original-title":[],"deposited":{"date-parts":[[2020,7,9]],"date-time":"2020-07-09T02:14:46Z","timestamp":1594260886000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.ijcai.org\/proceedings\/2020\/374"}},"subtitle":[],"proceedings-subject":"Artificial Intelligence Research Articles","short-title":[],"issued":{"date-parts":[[2020,7]]},"references-count":0,"URL":"https:\/\/doi.org\/10.24963\/ijcai.2020\/374","relation":{},"subject":[],"published":{"date-parts":[[2020,7]]}}}