{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T17:13:49Z","timestamp":1784135629831,"version":"3.55.0"},"publisher-location":"New York, NY, USA","reference-count":31,"publisher":"ACM","license":[{"start":{"date-parts":[[2024,10,25]],"date-time":"2024-10-25T00:00:00Z","timestamp":1729814400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,10,25]]},"DOI":"10.1145\/3704323.3704390","type":"proceedings-article","created":{"date-parts":[[2025,1,7]],"date-time":"2025-01-07T08:25:22Z","timestamp":1736238322000},"page":"334-339","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Enhancing Sample Efficiency in Online Reinforcement Learning via Policy-Guided Diffusion Models"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-6705-8422","authenticated-orcid":false,"given":"Yixuan","family":"Dong","sequence":"first","affiliation":[{"name":"Nanjing University of Science and Technology, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9865-9203","authenticated-orcid":false,"given":"Jun","family":"Zhang","sequence":"additional","affiliation":[{"name":"Nanjing University of Science and Technology, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,1,7]]},"reference":[{"key":"e_1_3_3_1_2_2","unstructured":"Rishabh Agarwal Dale Schuurmans and Mohammad Norouzi. An optimistic perspective on offline reinforcement learning. In International Conference on Machine Learning 2020."},{"key":"e_1_3_3_1_3_2","unstructured":"Marcin Andrychowicz Filip Wolski Alex Ray Jonas Schneider Rachel Fong Peter Welinder Bob McGrew Josh Tobin Pieter Abbeel and Wojciech Zaremba. Hindsight Experience Replay 2018."},{"key":"e_1_3_3_1_4_2","unstructured":"Philip J. Ball Laura Smith Ilya Kostrikov and Sergey Levine. Efficient online reinforcement learning with offline data 2023."},{"key":"e_1_3_3_1_5_2","unstructured":"Greg Brockman Vicki Cheung Ludwig Pettersson Jonas Schneider John Schulman Jie Tang and Wojciech Zaremba. Openai gym 2016."},{"key":"e_1_3_3_1_6_2","unstructured":"Xinyue Chen Che Wang Zijian Zhou and Keith W. Ross. Randomized ensembled double q-learning: Learning fast without a model. In International Conference on Learning Representations 2021."},{"key":"e_1_3_3_1_7_2","unstructured":"Daesol Cho Dongseok Shim and H Jin Kim. S2p: State-conditioned image synthesis for data augmentation in offline reinforcement learning. Advances in Neural Information Processing Systems volume 35 pages 11534\u201311546 2022."},{"key":"e_1_3_3_1_8_2","unstructured":"Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems volume 34 pages 8780\u20138794 2021."},{"key":"e_1_3_3_1_9_2","unstructured":"Pierluca D\u2019Oro Max Schwarzer Evgenii Nikishin Pierre-Luc Bacon Marc G Bellemare and Aaron Courville. Sample-efficient reinforcement learning by breaking the replay ratio barrier. In International Conference on Learning Representations 2023."},{"key":"e_1_3_3_1_10_2","unstructured":"Tuomas Haarnoja Aurick Zhou Pieter Abbeel and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International Conference on Machine Learning 2018."},{"key":"e_1_3_3_1_11_2","unstructured":"Danijar Hafner Timothy Lillicrap Jimmy Ba and Mohammad Norouzi. Dream to Control: Learning Behaviors by Latent Imagination. In International Conference on Learning Representations 2020."},{"key":"e_1_3_3_1_12_2","unstructured":"Danijar Hafner Timothy Lillicrap Mohammad Norouzi and Jimmy Ba. Mastering atari with discrete world models. In International Conference on Learning Representations 2021."},{"key":"e_1_3_3_1_13_2","doi-asserted-by":"crossref","unstructured":"Hado van Hasselt Arthur Guez and David Silver. Deep Reinforcement Learning with Double Q-learning. In Proceedings of the Thirtieth AAAI Conference on Arti\ufb01cial Intelligence 2016.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"e_1_3_3_1_14_2","unstructured":"Jonathan Ho Ajay Jain and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems volume 33 pages 6840\u20136851. Curran Associates Inc. 2020."},{"key":"e_1_3_3_1_15_2","unstructured":"Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance 2022."},{"key":"e_1_3_3_1_16_2","unstructured":"Matthew T. Jackson Michael T. Matthews Cong Lu Benjamin Ellis Shimon Whiteson and Jakob N. Foerster. Policy-Guided Diffusion 2024."},{"key":"e_1_3_3_1_17_2","unstructured":"Michael Janner Yilun Du Joshua B. Tenenbaum and Sergey Levine. Planning with diffusion for flexible behavior synthesis. In International Conference on Machine Learning 2022."},{"key":"e_1_3_3_1_18_2","unstructured":"Karol Kurach Mario Lucic Xiaohua Zhai Marcin Michalski and Sylvain Gelly. A large-scale study on regularization and normalization in GANs. In International Conference on Machine Learning 2019."},{"key":"e_1_3_3_1_19_2","unstructured":"Tero Karras Miika Aittala Timo Aila and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in Neural Information Processing Systems 2022."},{"key":"e_1_3_3_1_20_2","unstructured":"Sergey Levine Aviral Kumar George Tucker and Justin Fu. Offline reinforcement learning: Tutorial review and perspectives on open problems 2020."},{"key":"e_1_3_3_1_21_2","unstructured":"Zhixuan Liang Yao Mu Mingyu Ding Fei Ni Masayoshi Tomizuka and Ping Luo. AdaptDiffuser: Diffusion models as adaptive self-evolving planners. Proceedings of the 40th International Conference on Machine Learning volume 202 pages 20725\u201320745 2023."},{"key":"e_1_3_3_1_22_2","unstructured":"Cong Lu Philip J. Ball Yee Whye Teh and Jack Parker-Holder. Synthetic experience replay. In Conference on Neural Information Processing Systems 2023."},{"key":"e_1_3_3_1_23_2","doi-asserted-by":"crossref","unstructured":"Volodymyr Mnih Koray Kavukcuoglu David Silver Andrei A. Rusu Joel Veness Marc G. Bellemare Alex Graves et al. Human-level control through deep reinforcement learning. Nature volume 518 pages 529\u2013533 2015.","DOI":"10.1038\/nature14236"},{"key":"e_1_3_3_1_24_2","unstructured":"Marc Rigter Jun Yamada and Ingmar Posner. World Models via Policy-Guided Trajectory Diffusion. In Transactions on Machine Learning Research 2024."},{"key":"e_1_3_3_1_25_2","unstructured":"Tom Schaul John Quan Ioannis Antonoglou and David Silver. Prioritized Experience Replay. In International Conference on Learning Representations 2016."},{"key":"e_1_3_3_1_26_2","unstructured":"Jiaming Song Chenlin Meng and Stefano Ermon. Denoising diffusion implicit models 2022."},{"key":"e_1_3_3_1_27_2","unstructured":"Yang Song Jascha Sohl-Dickstein Diederik P. Kingma Abhishek Kumar Stefano Ermon and Ben Poole. Score-based generative modeling through stochastic differential equations 2021."},{"key":"e_1_3_3_1_28_2","unstructured":"Richard S. Sutton and Andrew G. Barto. Reinforcement learning: An introduction. MIT press 2018."},{"key":"e_1_3_3_1_29_2","unstructured":"Yuval Tassa Yotam Doron Alistair Muldal Tom Erez Yazhe Li Diego de Las Casas David Budden Abbas Abdolmaleki Josh Merel Andrew Lefrancq Timothy Lillicrap and Martin Riedmiller. DeepMind Control Suite 2018."},{"key":"e_1_3_3_1_30_2","doi-asserted-by":"crossref","unstructured":"Xu Wang Sen Wang Xingxing Liang Dawei Zhao Jincai Huang Xin Xu Bin Dai and Qiguang Miao. Deep Reinforcement Learning: A Survey. IEEE Transactions on Neural Networks and Learning Systems volume 35 pages 5064-5078 2022.","DOI":"10.1109\/TNNLS.2022.3207346"},{"key":"e_1_3_3_1_31_2","unstructured":"Long Yang Zhixiong Huang Fenghao Lei Yucun Zhong Yiming Yang Cong Fang Shiting Wen Binbin Zhou and Zhouchen Lin. Policy representation via diffusion probability model for reinforcement learning. 2023"},{"key":"e_1_3_3_1_32_2","unstructured":"Zhengbang Zhu Hanye Zhao Haoran He Yichao Zhong Shenyu Zhang Haoquan Guo Tingting Chen and Weinan Zhang. Diffusion Models for Reinforcement Learning: A Survey 2023."}],"event":{"name":"ICCPR 2024: 2024 13th International Conference on Computing and Pattern Recognition","location":"Tianjin China","acronym":"ICCPR 2024"},"container-title":["Proceedings of the 2024 13th International Conference on Computing and Pattern Recognition"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3704323.3704390","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3704323.3704390","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:18:05Z","timestamp":1750295885000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3704323.3704390"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,25]]},"references-count":31,"alternative-id":["10.1145\/3704323.3704390","10.1145\/3704323"],"URL":"https:\/\/doi.org\/10.1145\/3704323.3704390","relation":{},"subject":[],"published":{"date-parts":[[2024,10,25]]},"assertion":[{"value":"2025-01-07","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}