{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T10:01:31Z","timestamp":1777716091108,"version":"3.51.4"},"reference-count":40,"publisher":"SAGE Publications","issue":"9","license":[{"start":{"date-parts":[[2023,8,8]],"date-time":"2023-08-08T00:00:00Z","timestamp":1691452800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"DOI":"10.13039\/501100000923","name":"Australian Research Council","doi-asserted-by":"publisher","award":["200101049"],"award-info":[{"award-number":["200101049"]}],"id":[{"id":"10.13039\/501100000923","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of Robotics Research"],"published-print":{"date-parts":[[2024,8]]},"abstract":"<jats:p>\n                    Solving continuous Partially Observable Markov Decision Processes (POMDPs) is challenging, particularly for high-dimensional continuous action spaces. To alleviate this difficulty, we propose a new sampling-based online POMDP solver, called\n                    <jats:bold>\n                      <jats:italic>A<\/jats:italic>\n                    <\/jats:bold>\n                    <jats:italic>daptive<\/jats:italic>\n                    <jats:bold>\n                      <jats:italic>D<\/jats:italic>\n                    <\/jats:bold>\n                    <jats:italic>iscretization using<\/jats:italic>\n                    <jats:bold>\n                      <jats:italic>V<\/jats:italic>\n                    <\/jats:bold>\n                    <jats:italic>oronoi<\/jats:italic>\n                    <jats:bold>\n                      <jats:italic>T<\/jats:italic>\n                    <\/jats:bold>\n                    <jats:italic>rees (ADVT)<\/jats:italic>\n                    . It uses Monte Carlo Tree Search in combination with an adaptive discretization of the action space as well as optimistic optimization to efficiently sample high-dimensional continuous action spaces and compute the best action to perform. Specifically, we adaptively discretize the action space for each sampled belief using a hierarchical partition called\n                    <jats:italic>Voronoi tree<\/jats:italic>\n                    , which is a Binary Space Partitioning that implicitly maintains the partition of a cell as the Voronoi diagram of two points sampled from the cell. ADVT uses the estimated diameters of the cells to form an upper-confidence bound on the action value function within the cell, guiding the Monte Carlo Tree Search expansion and further discretization of the action space. This enables ADVT to better exploit local information with respect to the action value function, allowing faster identification of the most promising regions in the action space, compared to existing solvers. Voronoi trees keep the cost of partitioning and estimating the diameter of each cell low, even in high-dimensional spaces where many sampled points are required to cover the space well. ADVT additionally handles continuous observation spaces, by adopting an observation progressive widening strategy, along with a weighted particle representation of beliefs. Experimental results indicate that ADVT scales substantially better to high-dimensional continuous action spaces, compared to state-of-the-art methods.\n                  <\/jats:p>","DOI":"10.1177\/02783649231188984","type":"journal-article","created":{"date-parts":[[2023,8,8]],"date-time":"2023-08-08T21:16:16Z","timestamp":1691529376000},"page":"1283-1298","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":3,"title":["Adaptive Discretization using Voronoi Trees for Continuous POMDPs"],"prefix":"10.1177","volume":"43","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4698-5875","authenticated-orcid":false,"given":"Marcus","family":"Hoerger","sequence":"first","affiliation":[{"name":"School of Mathematics &amp; Physics, The University of Queensland, Queensland, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hanna","family":"Kurniawati","sequence":"additional","affiliation":[{"name":"School of Computing, Australian National University, Canberra, ACT, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dirk","family":"Kroese","sequence":"additional","affiliation":[{"name":"School of Mathematics &amp; Physics, The University of Queensland, Queensland, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nan","family":"Ye","sequence":"additional","affiliation":[{"name":"School of Mathematics &amp; Physics, The University of Queensland, Queensland, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2023,8,8]]},"reference":[{"key":"e_1_3_4_2_1","first-page":"4284","volume-title":"Firm: Feedback Controller-Based Information-State Roadmap. A Framework for Motion Planning under Uncertainty","author":"Agha-Mohammadi AA","year":"2011","unstructured":"Agha-Mohammadi AA, Chakravorty S, Amato NM (2011) Firm: Feedback Controller-Based Information-State Roadmap. A Framework for Motion Planning under Uncertainty. IROS, pp. 4284\u20134291."},{"key":"e_1_3_4_3_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1013689704352"},{"key":"e_1_3_4_4_1","doi-asserted-by":"publisher","DOI":"10.1177\/0278364914528255"},{"issue":"5","key":"e_1_3_4_5_1","first-page":"128","article-title":"X-armed bandits","volume":"12","author":"Bubeck S","year":"2011","unstructured":"Bubeck S, Munos R, Stoltz G, et al. (2011) X-armed bandits. Journal of Machine Learning Research 12(5): 128.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_4_6_1","volume-title":"Numerical Analysis","author":"Burden RL","year":"2016","unstructured":"Burden RL, Faires JD, Burden AM (2016) Numerical Analysis. 10th edition. Boston, MA: Cengage Learning.","edition":"10"},{"key":"e_1_3_4_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-25566-3_32"},{"key":"e_1_3_4_8_1","first-page":"3177","volume-title":"Information Particle Filter Tree: An Online Algorithm for POMDPs with Belief-Based Rewards on Continuous Domains","author":"Fischer J","year":"2020","unstructured":"Fischer J, Tas \u00d6S (2020) Information Particle Filter Tree: An Online Algorithm for POMDPs with Belief-Based Rewards on Continuous Domains. ICML. PMLR, pp. 3177\u20133187."},{"key":"e_1_3_4_9_1","doi-asserted-by":"crossref","unstructured":"Hoerger M Kurniawati H (2021) An on-line POMDP solver for continuous observation spaces. Proc. IEEE\/RSJ Int. Conference on Robotics and Automation (ICRA) May 2021.","DOI":"10.1109\/ICRA48506.2021.9560943"},{"key":"e_1_3_4_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2018.8593714"},{"key":"e_1_3_4_11_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-43089-4_18"},{"key":"e_1_3_4_12_1","volume-title":"Proc. Int. Workshop on the Algorithmic Foundations of Robotics","author":"Hoerger M","year":"2022","unstructured":"Hoerger M, Kurniawati H, Kroese D, et al. (2022) Adaptive discretization using voronoi trees for continuous-action pomdps. In: Proc. Int. Workshop on the Algorithmic Foundations of Robotics. Cornell University."},{"key":"e_1_3_4_13_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0004-3702(98)00023-X"},{"key":"e_1_3_4_14_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i06.6546"},{"key":"e_1_3_4_15_1","volume-title":"Tapir: A Software Toolkit for Approximating and Adapting POMDP Solutions Online","author":"Klimenko D","year":"2014","unstructured":"Klimenko D, Song J, Kurniawati H (2014) Tapir: A Software Toolkit for Approximating and Adapting POMDP Solutions Online. ACRA. https:\/\/github.com\/rdl-algorithm\/tapir."},{"key":"e_1_3_4_16_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-control-042920-092451"},{"key":"e_1_3_4_17_1","article-title":"An online POMDP solver for uncertainty planning in dynamic environment","author":"Kurniawati H","year":"2013","unstructured":"Kurniawati H, Yadav V (2013) An online POMDP solver for uncertainty planning in dynamic environment. Proc. Int. Symp. on Robotics Research. Online ahead of print.","journal-title":"Proc. Int. Symp. on Robotics Research"},{"key":"e_1_3_4_18_1","volume-title":"SARSOP: Efficient Point-Based POMDP Planning by Approximating Optimally Reachable Belief Spaces","author":"Kurniawati H","year":"2008","unstructured":"Kurniawati H, Hsu D, Lee WS (2008) SARSOP: Efficient Point-Based POMDP Planning by Approximating Optimally Reachable Belief Spaces. RSS."},{"key":"e_1_3_4_19_1","doi-asserted-by":"publisher","DOI":"10.1177\/0278364910386986"},{"key":"e_1_3_4_20_1","doi-asserted-by":"crossref","unstructured":"Lim MH Tomlin CJ Sunberg ZN (2021) Voronoi progressive widening: efficient online solvers for continuous state action and observation POMDPs. In: 60th IEEE Conference on Decision and Control (CDC) 2021 pp. 4493\u20134500.","DOI":"10.1109\/CDC45484.2021.9683490"},{"key":"e_1_3_4_21_1","doi-asserted-by":"publisher","DOI":"10.1137\/0311025"},{"key":"e_1_3_4_22_1","doi-asserted-by":"publisher","DOI":"10.1609\/icaps.v21i1.13484"},{"key":"e_1_3_4_23_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i13.17411"},{"key":"e_1_3_4_24_1","doi-asserted-by":"publisher","DOI":"10.1287\/moor.12.3.441"},{"key":"e_1_3_4_25_1","volume-title":"Point-Based Value Iteration: An Anytime Algorithm for POMDPs","author":"Pineau J","year":"2003","unstructured":"Pineau J, Gordon G, Thrun S (2003) Point-Based Value Iteration: An Anytime Algorithm for POMDPs. IJCAI."},{"key":"e_1_3_4_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2015.7139503"},{"key":"e_1_3_4_27_1","first-page":"2164","volume-title":"Advances in Neural Information Processing Systems","author":"Silver D","year":"2010","unstructured":"Silver D, Veness J (2010) Monte Carlo planning in large POMDPs. Advances in Neural Information Processing Systems, pp. 2164\u20132172."},{"key":"e_1_3_4_28_1","doi-asserted-by":"publisher","DOI":"10.1287\/opre.32.6.1296"},{"key":"e_1_3_4_29_1","volume-title":"Point-Based POMDP Algorithms: Improved Analysis and Implementation","author":"Smith T","year":"2005","unstructured":"Smith T, Simmons R (2005) Point-Based POMDP Algorithms: Improved Analysis and Implementation. UAI."},{"key":"e_1_3_4_30_1","volume-title":"The Optimal Control of Partially Observable Markov Decision Processes","author":"Sondik EJ","year":"1971","unstructured":"Sondik EJ (1971) The Optimal Control of Partially Observable Markov Decision Processes. PhD Thesis. Stanford, CA: JSTOR."},{"issue":"1","key":"e_1_3_4_31_1","first-page":"104","article-title":"High-frequency replanning under uncertainty using parallel sampling-based motion planning","volume":"31","author":"Sun W","year":"2015","unstructured":"Sun W, Patil S, Alterovitz R (2015) High-frequency replanning under uncertainty using parallel sampling-based motion planning. IEEE TRO 31(1): 104\u2013116.","journal-title":"IEEE TRO"},{"key":"e_1_3_4_32_1","doi-asserted-by":"publisher","DOI":"10.1609\/icaps.v28i1.13882"},{"key":"e_1_3_4_33_1","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton RS","year":"2018","unstructured":"Sutton RS, Barto AG (2018) Reinforcement Learning: An Introduction. MIT press."},{"key":"e_1_3_4_34_1","volume-title":"Zooming for Efficient Model-free Reinforcement Learning in Metric Spaces","author":"Touati A","year":"2020","unstructured":"Touati A, Taiga AA, Bellemare MG (2020) Zooming for Efficient Model-free Reinforcement Learning in Metric Spaces. arXiv preprint 2003."},{"key":"e_1_3_4_35_1","first-page":"19","volume-title":"Stochastic Simultaneous Optimistic Optimization","author":"Valko M","year":"2013","unstructured":"Valko M, Carpentier A, Munos R (2013) Stochastic Simultaneous Optimistic Optimization. ICML, pp. 19\u201327."},{"key":"e_1_3_4_36_1","doi-asserted-by":"publisher","DOI":"10.1177\/0278364911406562"},{"key":"e_1_3_4_37_1","doi-asserted-by":"publisher","DOI":"10.1177\/0278364912456319"},{"key":"e_1_3_4_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3412815.3416885"},{"key":"e_1_3_4_39_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00992698"},{"key":"e_1_3_4_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/BFb0038202"},{"key":"e_1_3_4_41_1","doi-asserted-by":"publisher","DOI":"10.1613\/jair.5328"}],"container-title":["The International Journal of Robotics Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/02783649231188984","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/02783649231188984","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/02783649231188984","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/02783649231188984","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T10:17:19Z","timestamp":1777457839000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/02783649231188984"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,8]]},"references-count":40,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2024,8]]}},"alternative-id":["10.1177\/02783649231188984"],"URL":"https:\/\/doi.org\/10.1177\/02783649231188984","relation":{},"ISSN":["0278-3649","1741-3176"],"issn-type":[{"value":"0278-3649","type":"print"},{"value":"1741-3176","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,8]]}}}