{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,12]],"date-time":"2025-06-12T04:25:30Z","timestamp":1749702330228,"version":"3.40.5"},"reference-count":0,"publisher":"IOS Press","isbn-type":[{"type":"print","value":"9781643684369"},{"type":"electronic","value":"9781643684376"}],"license":[{"start":{"date-parts":[[2023,9,28]],"date-time":"2023-09-28T00:00:00Z","timestamp":1695859200000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,9,28]]},"abstract":"<jats:p>We consider the challenging scenario of contextual bandits with continuous actions and large context spaces. This is an increasingly important application area in personalised healthcare where an agent is requested to make dosing decisions based on a patient\u2019s single image scan. In this paper, we first adapt a reinforcement learning (RL) algorithm for continuous control to outperform contextual bandit algorithms specifically hand-crafted for continuous action spaces. We empirically demonstrate this on a suite of standard benchmark datasets for vector contexts. Secondly, we demonstrate that our RL agent can generalise problems with continuous actions to large context spaces, providing results that outperform previous methods on image contexts. Thirdly, we introduce a new contextual bandits test domain with multi-dimensional continuous action space and image contexts which existing tree-based methods cannot handle. We provide initial results with our RL agent.<\/jats:p>","DOI":"10.3233\/faia230320","type":"book-chapter","created":{"date-parts":[[2023,9,29]],"date-time":"2023-09-29T09:07:00Z","timestamp":1695978420000},"source":"Crossref","is-referenced-by-count":2,"title":["Reinforcement Learning for Bandits with Continuous Actions and Large Context Spaces"],"prefix":"10.3233","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9052-6919","authenticated-orcid":false,"given":"Paul","family":"Duckworth","sequence":"first","affiliation":[{"name":"Oxford Robotics Institute, Department of Engineering Science, University of Oxford"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4672-5683","authenticated-orcid":false,"given":"Katherine A.","family":"Vallis","sequence":"additional","affiliation":[{"name":"Department of Oncology, University of Oxford"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0862-331X","authenticated-orcid":false,"given":"Bruno","family":"Lacerda","sequence":"additional","affiliation":[{"name":"Oxford Robotics Institute, Department of Engineering Science, University of Oxford"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7556-6098","authenticated-orcid":false,"given":"Nick","family":"Hawes","sequence":"additional","affiliation":[{"name":"Oxford Robotics Institute, Department of Engineering Science, University of Oxford"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"7437","container-title":["Frontiers in Artificial Intelligence and Applications","ECAI 2023"],"original-title":[],"link":[{"URL":"https:\/\/ebooks.iospress.nl\/pdf\/doi\/10.3233\/FAIA230320","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,29]],"date-time":"2023-09-29T09:07:02Z","timestamp":1695978422000},"score":1,"resource":{"primary":{"URL":"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/FAIA230320"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,28]]},"ISBN":["9781643684369","9781643684376"],"references-count":0,"URL":"https:\/\/doi.org\/10.3233\/faia230320","relation":{},"ISSN":["0922-6389","1879-8314"],"issn-type":[{"type":"print","value":"0922-6389"},{"type":"electronic","value":"1879-8314"}],"subject":[],"published":{"date-parts":[[2023,9,28]]}}}