{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T02:44:59Z","timestamp":1777085099835,"version":"3.51.4"},"publisher-location":"New York, NY, USA","reference-count":32,"publisher":"ACM","license":[{"start":{"date-parts":[[2023,3,17]],"date-time":"2023-03-17T00:00:00Z","timestamp":1679011200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,3,17]]},"DOI":"10.1145\/3590003.3590058","type":"proceedings-article","created":{"date-parts":[[2023,5,29]],"date-time":"2023-05-29T18:22:56Z","timestamp":1685384576000},"page":"309-315","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Multiple Frequency Bands Temporal State Representation for Deep Reinforcement Learning"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-5171-9553","authenticated-orcid":false,"given":"Che","family":"Wang","sequence":"first","affiliation":[{"name":"Jilin University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8658-9447","authenticated-orcid":false,"given":"Jifeng","family":"Hu","sequence":"additional","affiliation":[{"name":"Jilin University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-5017-5430","authenticated-orcid":false,"given":"Fuhu","family":"Song","sequence":"additional","affiliation":[{"name":"Jilin University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-8776-0186","authenticated-orcid":false,"given":"Jiao","family":"Huang","sequence":"additional","affiliation":[{"name":"Jilin University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-0598-2082","authenticated-orcid":false,"given":"Zixuan","family":"Yang","sequence":"additional","affiliation":[{"name":"Jilin University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-6108-5850","authenticated-orcid":false,"given":"Yusen","family":"Wang","sequence":"additional","affiliation":[{"name":"Jilin University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,5,29]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Training: Unity ML-Agents. In International Work-Conference on the Interplay Between Natural and Artificial Computation","author":"Alm\u00f3n-Manzano Laura","unstructured":"Laura Alm\u00f3n-Manzano , Rafael Pastor-Vargas , and Jos\u00e9 Manuel\u00a0Cuadra Troncoso . 2022. Deep Reinforcement Learning in Agents \u2019 Training: Unity ML-Agents. In International Work-Conference on the Interplay Between Natural and Artificial Computation . Springer , 391\u2013400. Laura Alm\u00f3n-Manzano, Rafael Pastor-Vargas, and Jos\u00e9 Manuel\u00a0Cuadra Troncoso. 2022. Deep Reinforcement Learning in Agents\u2019 Training: Unity ML-Agents. In International Work-Conference on the Interplay Between Natural and Artificial Computation. Springer, 391\u2013400."},{"key":"e_1_3_2_1_2_1","volume-title":"Emergent tool use from multi-agent autocurricula. arXiv preprint arXiv:1909.07528","author":"Baker Bowen","year":"2019","unstructured":"Bowen Baker , Ingmar Kanitscheider , Todor Markov , Yi Wu , Glenn Powell , Bob McGrew , and Igor Mordatch . 2019. Emergent tool use from multi-agent autocurricula. arXiv preprint arXiv:1909.07528 ( 2019 ). Bowen Baker, Ingmar Kanitscheider, Todor Markov, Yi Wu, Glenn Powell, Bob McGrew, and Igor Mordatch. 2019. Emergent tool use from multi-agent autocurricula. arXiv preprint arXiv:1909.07528 (2019)."},{"key":"e_1_3_2_1_3_1","volume-title":"Coberl: Contrastive bert for reinforcement learning. arXiv preprint arXiv:2107.05431","author":"Banino Andrea","year":"2021","unstructured":"Andrea Banino , Adri\u00e0\u00a0Puidomenech Badia , Jacob Walker , Tim Scholtes , Jovana Mitrovic , and Charles Blundell . 2021 . Coberl: Contrastive bert for reinforcement learning. arXiv preprint arXiv:2107.05431 (2021). Andrea Banino, Adri\u00e0\u00a0Puidomenech Badia, Jacob Walker, Tim Scholtes, Jovana Mitrovic, and Charles Blundell. 2021. Coberl: Contrastive bert for reinforcement learning. arXiv preprint arXiv:2107.05431 (2021)."},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jksuci.2018.06.002"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1177\/1475921720935585"},{"key":"e_1_3_2_1_6_1","volume-title":"An introduction to deep reinforcement learning. Foundations and Trends\u00ae in Machine Learning 11, 3-4","author":"Fran\u00e7ois-Lavet Vincent","year":"2018","unstructured":"Vincent Fran\u00e7ois-Lavet , Peter Henderson , Riashat Islam , Marc\u00a0 G Bellemare , Joelle Pineau , 2018. An introduction to deep reinforcement learning. Foundations and Trends\u00ae in Machine Learning 11, 3-4 ( 2018 ), 219\u2013354. Vincent Fran\u00e7ois-Lavet, Peter Henderson, Riashat Islam, Marc\u00a0G Bellemare, Joelle Pineau, 2018. An introduction to deep reinforcement learning. Foundations and Trends\u00ae in Machine Learning 11, 3-4 (2018), 219\u2013354."},{"key":"e_1_3_2_1_7_1","volume-title":"International conference on machine learning. PMLR","author":"Haarnoja Tuomas","year":"2018","unstructured":"Tuomas Haarnoja , Aurick Zhou , Pieter Abbeel , and Sergey Levine . 2018 . Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor . In International conference on machine learning. PMLR , 1861\u20131870. Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. 2018. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning. PMLR, 1861\u20131870."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2022.3142822"},{"key":"e_1_3_2_1_9_1","volume-title":"Tuning Proximal Policy Optimization Algorithm in Maze Solving with ML-Agents. In International Conference on Advances in Computing and Data Sciences. Springer, 248\u2013262","author":"Hung Phan\u00a0Thanh","year":"2022","unstructured":"Phan\u00a0Thanh Hung , Mac Duy\u00a0Dan Truong , and Phan\u00a0Duy Hung . 2022 . Tuning Proximal Policy Optimization Algorithm in Maze Solving with ML-Agents. In International Conference on Advances in Computing and Data Sciences. Springer, 248\u2013262 . Phan\u00a0Thanh Hung, Mac Duy\u00a0Dan Truong, and Phan\u00a0Duy Hung. 2022. Tuning Proximal Policy Optimization Algorithm in Maze Solving with ML-Agents. In International Conference on Advances in Computing and Data Sciences. Springer, 248\u2013262."},{"key":"e_1_3_2_1_10_1","volume-title":"Unity: A general platform for intelligent agents. arXiv preprint arXiv:1809.02627","author":"Juliani Arthur","year":"2020","unstructured":"Arthur Juliani , Vincent-Pierre Berges , Ervin Teng , Andrew Cohen , Jonathan Harper , Chris Elion , Chris Goy , Yuan Gao , Hunter Henry , Marwan Mattar , and Danny Lange . 2020 . Unity: A general platform for intelligent agents. arXiv preprint arXiv:1809.02627 (2020). Arthur Juliani, Vincent-Pierre Berges, Ervin Teng, Andrew Cohen, Jonathan Harper, Chris Elion, Chris Goy, Yuan Gao, Hunter Henry, Marwan Mattar, and Danny Lange. 2020. Unity: A general platform for intelligent agents. arXiv preprint arXiv:1809.02627 (2020)."},{"key":"e_1_3_2_1_11_1","volume-title":"Model-based reinforcement learning for atari. arXiv preprint arXiv:1903.00374","author":"Kaiser Lukasz","year":"2019","unstructured":"Lukasz Kaiser , Mohammad Babaeizadeh , Piotr Milos , Blazej Osinski , Roy\u00a0 H Campbell , Konrad Czechowski , Dumitru Erhan , Chelsea Finn , Piotr Kozakowski , Sergey Levine , 2019. Model-based reinforcement learning for atari. arXiv preprint arXiv:1903.00374 ( 2019 ). Lukasz Kaiser, Mohammad Babaeizadeh, Piotr Milos, Blazej Osinski, Roy\u00a0H Campbell, Konrad Czechowski, Dumitru Erhan, Chelsea Finn, Piotr Kozakowski, Sergey Levine, 2019. Model-based reinforcement learning for atari. arXiv preprint arXiv:1903.00374 (2019)."},{"key":"e_1_3_2_1_12_1","volume-title":"International conference on learning representations.","author":"Kapturowski Steven","year":"2018","unstructured":"Steven Kapturowski , Georg Ostrovski , John Quan , Remi Munos , and Will Dabney . 2018 . Recurrent experience replay in distributed reinforcement learning . In International conference on learning representations. Steven Kapturowski, Georg Ostrovski, John Quan, Remi Munos, and Will Dabney. 2018. Recurrent experience replay in distributed reinforcement learning. In International conference on learning representations."},{"key":"e_1_3_2_1_13_1","volume-title":"International conference on learning representations.","author":"Kapturowski Steven","year":"2018","unstructured":"Steven Kapturowski , Georg Ostrovski , John Quan , Remi Munos , and Will Dabney . 2018 . Recurrent experience replay in distributed reinforcement learning . In International conference on learning representations. Steven Kapturowski, Georg Ostrovski, John Quan, Remi Munos, and Will Dabney. 2018. Recurrent experience replay in distributed reinforcement learning. In International conference on learning representations."},{"key":"e_1_3_2_1_14_1","volume-title":"Fourier analysis","author":"K\u00f6rner Thomas\u00a0William","unstructured":"Thomas\u00a0William K\u00f6rner . 2022. Fourier analysis . Cambridge university press . Thomas\u00a0William K\u00f6rner. 2022. Fourier analysis. Cambridge university press."},{"key":"e_1_3_2_1_15_1","volume-title":"Recurrent reinforcement learning: a hybrid approach. arXiv preprint arXiv:1509.03044","author":"Li Xiujun","year":"2015","unstructured":"Xiujun Li , Lihong Li , Jianfeng Gao , Xiaodong He , Jianshu Chen , Li Deng , and Ji He. 2015. Recurrent reinforcement learning: a hybrid approach. arXiv preprint arXiv:1509.03044 ( 2015 ). Xiujun Li, Lihong Li, Jianfeng Gao, Xiaodong He, Jianshu Chen, Li Deng, and Ji He. 2015. Recurrent reinforcement learning: a hybrid approach. arXiv preprint arXiv:1509.03044 (2015)."},{"key":"e_1_3_2_1_16_1","volume-title":"Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971","author":"Lillicrap P","year":"2015","unstructured":"Timothy\u00a0 P Lillicrap , Jonathan\u00a0 J Hunt , Alexander Pritzel , Nicolas Heess , Tom Erez , Yuval Tassa , David Silver , and Daan Wierstra . 2015. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 ( 2015 ). Timothy\u00a0P Lillicrap, Jonathan\u00a0J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2015. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 (2015)."},{"key":"e_1_3_2_1_17_1","volume-title":"Juewu-mc: Playing minecraft with sample-efficient hierarchical reinforcement learning. arXiv preprint arXiv:2112.04907","author":"Lin Zichuan","year":"2021","unstructured":"Zichuan Lin , Junyou Li , Jianing Shi , Deheng Ye , Qiang Fu , and Wei Yang . 2021 . Juewu-mc: Playing minecraft with sample-efficient hierarchical reinforcement learning. arXiv preprint arXiv:2112.04907 (2021). Zichuan Lin, Junyou Li, Jianing Shi, Deheng Ye, Qiang Fu, and Wei Yang. 2021. Juewu-mc: Playing minecraft with sample-efficient hierarchical reinforcement learning. arXiv preprint arXiv:2112.04907 (2021)."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/IV51971.2022.9827073"},{"key":"e_1_3_2_1_19_1","volume-title":"Agent inspired trading using recurrent reinforcement learning and lstm neural networks. arXiv preprint arXiv:1707.07338","author":"W Lu.","year":"2017","unstructured":"David\u00a0 W Lu. 2017. Agent inspired trading using recurrent reinforcement learning and lstm neural networks. arXiv preprint arXiv:1707.07338 ( 2017 ). David\u00a0W Lu. 2017. Agent inspired trading using recurrent reinforcement learning and lstm neural networks. arXiv preprint arXiv:1707.07338 (2017)."},{"key":"e_1_3_2_1_20_1","volume-title":"Sample efficient deep reinforcement learning via uncertainty estimation. arXiv preprint arXiv:2201.01666","author":"Mai Vincent","year":"2022","unstructured":"Vincent Mai , Kaustubh Mani , and Liam Paull . 2022. Sample efficient deep reinforcement learning via uncertainty estimation. arXiv preprint arXiv:2201.01666 ( 2022 ). Vincent Mai, Kaustubh Mani, and Liam Paull. 2022. Sample efficient deep reinforcement learning via uncertainty estimation. arXiv preprint arXiv:2201.01666 (2022)."},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-36802-9_25"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3461598.3461608"},{"key":"e_1_3_2_1_23_1","volume-title":"Conference on Robot Learning. PMLR, 1634\u20131644","author":"Raffin Antonin","year":"2022","unstructured":"Antonin Raffin , Jens Kober , and Freek Stulp . 2022 . Smooth exploration for robotic reinforcement learning . In Conference on Robot Learning. PMLR, 1634\u20131644 . Antonin Raffin, Jens Kober, and Freek Stulp. 2022. Smooth exploration for robotic reinforcement learning. In Conference on Robot Learning. PMLR, 1634\u20131644."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.424"},{"key":"e_1_3_2_1_25_1","volume-title":"Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347","author":"Schulman John","year":"2017","unstructured":"John Schulman , Filip Wolski , Prafulla Dhariwal , Alec Radford , and Oleg Klimov . 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 ( 2017 ). John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)."},{"key":"e_1_3_2_1_26_1","volume-title":"Mastering the game of go without human knowledge. nature 550, 7676","author":"Silver David","year":"2017","unstructured":"David Silver , Julian Schrittwieser , Karen Simonyan , Ioannis Antonoglou , Aja Huang , Arthur Guez , Thomas Hubert , Lucas Baker , Matthew Lai , Adrian Bolton , 2017. Mastering the game of go without human knowledge. nature 550, 7676 ( 2017 ), 354\u2013359. David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, 2017. Mastering the game of go without human knowledge. nature 550, 7676 (2017), 354\u2013359."},{"key":"e_1_3_2_1_27_1","volume-title":"Reinforcement learning: An introduction","author":"Sutton S","unstructured":"Richard\u00a0 S Sutton and Andrew\u00a0 G Barto . 2018. Reinforcement learning: An introduction . MIT press . Richard\u00a0S Sutton and Andrew\u00a0G Barto. 2018. Reinforcement learning: An introduction. MIT press."},{"key":"e_1_3_2_1_28_1","volume-title":"International Conference on Machine Learning. PMLR, 22270\u201322283","author":"Villaflor R","year":"2022","unstructured":"Adam\u00a0 R Villaflor , Zhe Huang , Swapnil Pande , John\u00a0 M Dolan , and Jeff Schneider . 2022 . Addressing optimism bias in sequence modeling for reinforcement learning . In International Conference on Machine Learning. PMLR, 22270\u201322283 . Adam\u00a0R Villaflor, Zhe Huang, Swapnil Pande, John\u00a0M Dolan, and Jeff Schneider. 2022. Addressing optimism bias in sequence modeling for reinforcement learning. In International Conference on Machine Learning. PMLR, 22270\u201322283."},{"key":"e_1_3_2_1_29_1","volume-title":"A review on weed detection using ground-based machine vision and image processing techniques. Computers and electronics in agriculture 158","author":"Wang Aichen","year":"2019","unstructured":"Aichen Wang , Wen Zhang , and Xinhua Wei . 2019. A review on weed detection using ground-based machine vision and image processing techniques. Computers and electronics in agriculture 158 ( 2019 ), 226\u2013240. Aichen Wang, Wen Zhang, and Xinhua Wei. 2019. A review on weed detection using ground-based machine vision and image processing techniques. Computers and electronics in agriculture 158 (2019), 226\u2013240."},{"key":"e_1_3_2_1_30_1","volume-title":"Deep reinforcement learning: a survey","author":"Wang Xu","year":"2022","unstructured":"Xu Wang , Sen Wang , Xingxing Liang , Dawei Zhao , Jincai Huang , Xin Xu , Bin Dai , and Qiguang Miao . 2022. Deep reinforcement learning: a survey . IEEE Transactions on Neural Networks and Learning Systems ( 2022 ). Xu Wang, Sen Wang, Xingxing Liang, Dawei Zhao, Jincai Huang, Xin Xu, Bin Dai, and Qiguang Miao. 2022. Deep reinforcement learning: a survey. IEEE Transactions on Neural Networks and Learning Systems (2022)."},{"key":"e_1_3_2_1_31_1","volume-title":"Deep reinforcement learning in medical imaging: A literature review. Medical image analysis 73","author":"Zhou S\u00a0Kevin","year":"2021","unstructured":"S\u00a0Kevin Zhou , Hoang\u00a0Ngan Le , Khoa Luu , Hien\u00a0 V Nguyen , and Nicholas Ayache . 2021. Deep reinforcement learning in medical imaging: A literature review. Medical image analysis 73 ( 2021 ), 102193. S\u00a0Kevin Zhou, Hoang\u00a0Ngan Le, Khoa Luu, Hien\u00a0V Nguyen, and Nicholas Ayache. 2021. Deep reinforcement learning in medical imaging: A literature review. Medical image analysis 73 (2021), 102193."},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.26599\/TST.2021.9010012"}],"event":{"name":"CACML 2023: 2023 2nd Asia Conference on Algorithms, Computing and Machine Learning","location":"Shanghai China","acronym":"CACML 2023"},"container-title":["Proceedings of the 2023 2nd Asia Conference on Algorithms, Computing and Machine Learning"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3590003.3590058","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3590003.3590058","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T18:09:16Z","timestamp":1750183756000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3590003.3590058"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,17]]},"references-count":32,"alternative-id":["10.1145\/3590003.3590058","10.1145\/3590003"],"URL":"https:\/\/doi.org\/10.1145\/3590003.3590058","relation":{},"subject":[],"published":{"date-parts":[[2023,3,17]]},"assertion":[{"value":"2023-05-29","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}