{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,7]],"date-time":"2026-08-07T16:40:59Z","timestamp":1786120859900,"version":"build-2736575974"},"publisher-location":"New York, NY, USA","reference-count":64,"publisher":"ACM","license":[{"start":{"date-parts":[[2026,8,8]],"date-time":"2026-08-08T00:00:00Z","timestamp":1786147200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"funder":[{"name":"Noncommunicable Chronic Diseases-National Science and Technology Major Project","award":["No. 2024ZD0532400"],"award-info":[{"award-number":["No. 2024ZD0532400"]}]},{"name":"Noncommunicable Chronic Diseases-National Science and Technology Major Project","award":["No. 2024ZD0532403"],"award-info":[{"award-number":["No. 2024ZD0532403"]}]},{"name":"NSF","award":["IIS-2106972"],"award-info":[{"award-number":["IIS-2106972"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2026,8,9]]},"DOI":"10.1145\/3770855.3817803","type":"proceedings-article","created":{"date-parts":[[2026,8,6]],"date-time":"2026-08-06T17:15:43Z","timestamp":1786036543000},"page":"6546-6557","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["RAPO: Expanding Exploration for LLM Agents via Retrieval-Augmented Policy Optimization"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-3994-083X","authenticated-orcid":false,"given":"Siwei","family":"Zhang","sequence":"first","affiliation":[{"name":"Shanghai Key Laboratory of Data Science, College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8575-5415","authenticated-orcid":false,"given":"Yun","family":"Xiong","sequence":"additional","affiliation":[{"name":"Shanghai Key Laboratory of Data Science, College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-6156-4811","authenticated-orcid":false,"given":"Xi","family":"Chen","sequence":"additional","affiliation":[{"name":"Shanghai Key Laboratory of Data Science, College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-0117-8847","authenticated-orcid":false,"given":"Zi'an","family":"Jia","sequence":"additional","affiliation":[{"name":"Shanghai Key Laboratory of Data Science, College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7808-9768","authenticated-orcid":false,"given":"Renhong","family":"Huang","sequence":"additional","affiliation":[{"name":"DCD Lab, Zhejiang University, Hangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2973-1889","authenticated-orcid":false,"given":"Jiarong","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Management, Fudan University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2111-7617","authenticated-orcid":false,"given":"Jiawei","family":"Zhang","sequence":"additional","affiliation":[{"name":"University of California, Davis, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,8,8]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"V4, 5","author":"Chen Jianlv","year":"2024","unstructured":"Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. arXiv preprint arXiv:2402.03216, V4, 5 (2024)."},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"crossref","unstructured":"Mingyang Chen Linzhuang Sun Tianpeng Li Haoze Sun Yijie Zhou Chenzheng Zhu Haofen Wang Jeff Z Pan Wen Zhang Huajun Chen et al. 2025b. Learning to reason with search for llms via reinforcement learning. arXiv preprint arXiv:2503.19470 (2025).","DOI":"10.52202\/085713-2858"},{"key":"e_1_3_2_1_3_1","volume-title":"Toward Effective Tool-Integrated Reasoning via Self-Evolved Preference Learning. arXiv preprint arXiv:2509.23285","author":"Chen Yifei","year":"2025","unstructured":"Yifei Chen, Guanting Dong, and Zhicheng Dou. 2025a. Toward Effective Tool-Integrated Reasoning via Self-Evolved Preference Learning. arXiv preprint arXiv:2509.23285 (2025)."},{"key":"e_1_3_2_1_4_1","volume-title":"Zhenliang Zhang, and Furu Wei.","author":"Cheng Daixuan","year":"2025","unstructured":"Daixuan Cheng, Shaohan Huang, Xuekai Zhu, Bo Dai, Wayne Xin Zhao, Zhenliang Zhang, and Furu Wei. 2025. Reasoning with exploration: An entropy perspective. arXiv preprint arXiv:2506.14758 (2025)."},{"key":"e_1_3_2_1_5_1","unstructured":"Karl Cobbe Vineet Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun Lukasz Kaiser Matthias Plappert Jerry Tworek Jacob Hilton Reiichiro Nakano et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 (2021)."},{"key":"e_1_3_2_1_6_1","volume-title":"Wayne Xin Zhao, and Ji-Rong Wen","author":"Deng Jia","year":"2025","unstructured":"Jia Deng, Jie Chen, Zhipeng Chen, Daixuan Cheng, Fei Bai, Beichen Zhang, Yinqian Min, Yanzipeng Gao, Wayne Xin Zhao, and Ji-Rong Wen. 2025. From trial-and-error to improvement: A systematic analysis of llm exploration mechanisms in rlvr. arXiv preprint arXiv:2508.07534 (2025)."},{"key":"e_1_3_2_1_7_1","volume-title":"The International World Wide Web Conference","author":"Dong Guanting","year":"2026","unstructured":"Guanting Dong, Licheng Bao, Zhongyuan Wang, Kangzhi Zhao, Xiaoxi Li, Jiajie Jin, Jinghan Yang, Hangyu Mao, Fuzheng Zhang, Kun Gai, et al., 2026 a. Agentic entropy-balanced policy optimization. The International World Wide Web Conference (2026)."},{"key":"e_1_3_2_1_8_1","volume-title":"Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learning. arXiv preprint arXiv:2505.16410","author":"Dong Guanting","year":"2025","unstructured":"Guanting Dong, Yifei Chen, Xiaoxi Li, Jiajie Jin, Hongjin Qian, Yutao Zhu, Hangyu Mao, Guorui Zhou, Zhicheng Dou, and Ji-Rong Wen. 2025. Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learning. arXiv preprint arXiv:2505.16410 (2025)."},{"key":"e_1_3_2_1_9_1","volume-title":"The International Conference on Learning Representations","author":"Dong Guanting","year":"2026","unstructured":"Guanting Dong, Hangyu Mao, Kai Ma, Licheng Bao, Yifei Chen, Zhongyuan Wang, Zhongxia Chen, Jiazhen Du, Huiyang Wang, Fuzheng Zhang, et al., 2026 b. Agentic reinforced policy optimization. The International Conference on Learning Representations (2026)."},{"key":"e_1_3_2_1_10_1","unstructured":"Martin Ester Hans-Peter Kriegel J\u00f6rg Sander Xiaowei Xu et al. 1996. A density-based algorithm for discovering clusters in large spatial databases with noise. In kdd Vol. V96. 226-231."},{"key":"e_1_3_2_1_11_1","unstructured":"Jinyuan Fang Yanwen Peng Xi Zhang Yingxu Wang Xinhao Yi Guibin Zhang Yi Xu Bin Wu Siwei Liu Zihao Li et al. 2025. A comprehensive survey of self-evolving ai agents: A new paradigm bridging foundation models and lifelong agentic systems. arXiv preprint arXiv:2508.07407 (2025)."},{"key":"e_1_3_2_1_12_1","volume-title":"Retool: Reinforcement learning for strategic tool use in llms. arXiv preprint arXiv:2504.11536","author":"Feng Jiazhan","year":"2025","unstructured":"Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. 2025a. Retool: Reinforcement learning for strategic tool use in llms. arXiv preprint arXiv:2504.11536 (2025)."},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.52202\/085713-1544"},{"key":"e_1_3_2_1_14_1","volume-title":"Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars. arXiv preprint arXiv:2503.01307","author":"Gandhi Kanishk","year":"2025","unstructured":"Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. 2025. Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars. arXiv preprint arXiv:2503.01307 (2025)."},{"key":"e_1_3_2_1_15_1","unstructured":"Aaron Grattafiori Abhimanyu Dubey Abhinav Jauhri Abhinav Pandey Abhishek Kadian Ahmad Al-Dahle Aiesha Letman Akhil Mathur Alan Schelten Alex Vaughan et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 (2024)."},{"key":"e_1_3_2_1_16_1","volume-title":"Rui Yan, Chaojie Wang, Peng Cheng, Xiaoyu Zhang, Fuxiang Zhang, Jiacheng Xu, Wei Shen, et al.","author":"He Jujie","year":"2025","unstructured":"Jujie He, Jiacai Liu, Chris Yuhao Liu, Rui Yan, Chaojie Wang, Peng Cheng, Xiaoyu Zhang, Fuxiang Zhang, Jiacheng Xu, Wei Shen, et al., 2025. Skywork open reasoner 1 technical report. arXiv preprint arXiv:2505.22312 (2025)."},{"key":"e_1_3_2_1_17_1","volume-title":"Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874","author":"Hendrycks Dan","year":"2021","unstructured":"Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874 (2021)."},{"key":"e_1_3_2_1_18_1","volume-title":"Saku Sugawara, and Akiko Aizawa.","author":"Ho Xanh","year":"2020","unstructured":"Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps. arXiv preprint arXiv:2011.01060 (2020)."},{"key":"e_1_3_2_1_19_1","volume-title":"The International Conference on Learning Representations","author":"Ji Yuxiang","year":"2026","unstructured":"Yuxiang Ji, Ziyu Ma, Yong Wang, Guanhua Chen, Xiangxiang Chu, and Liaoni Wu. 2026. Tree search for llm agent reinforcement learning. The International Conference on Learning Representations (2026)."},{"key":"e_1_3_2_1_20_1","volume-title":"Verltool: Towards holistic agentic reinforcement learning with tool use. arXiv preprint arXiv:2509.01055","author":"Jiang Dongfu","year":"2025","unstructured":"Dongfu Jiang, Yi Lu, Zhuofeng Li, Zhiheng Lyu, Ping Nie, Haozhe Wang, Alex Su, Hui Chen, Kai Zou, Chao Du, et al., 2025. Verltool: Towards holistic agentic reinforcement learning with tool use. arXiv preprint arXiv:2509.01055 (2025)."},{"key":"e_1_3_2_1_21_1","volume-title":"Search-r1: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516","author":"Jin Bowen","year":"2025","unstructured":"Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. Search-r1: Training llms to reason and leverage search engines with reinforcement learning. arXiv preprint arXiv:2503.09516 (2025)."},{"key":"e_1_3_2_1_22_1","volume-title":"Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih.","author":"Karpukhin Vladimir","year":"2020","unstructured":"Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick SH Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense Passage Retrieval for Open-Domain Question Answering.. In EMNLP (1). 6769-6781."},{"key":"e_1_3_2_1_23_1","unstructured":"Jack Lanchantin Angelica Chen Janice Lan Xian Li Swarnadeep Saha Tianlu Wang Jing Xu Ping Yu Weizhe Yuan Jason E Weston et al. 2025. Bridging Offline and Online Reinforcement Learning for LLMs. arXiv preprint arXiv:2506.21495 (2025)."},{"key":"e_1_3_2_1_24_1","unstructured":"Kuan Li Zhongwang Zhang Huifeng Yin Liwen Zhang Litu Ou Jialong Wu Wenbiao Yin Baixuan Li Zhengwei Tao Xinyu Wang et al. 2025c. WebSailor: Navigating Super-human Reasoning for Web Agent. arXiv preprint arXiv:2507.02592 (2025)."},{"key":"e_1_3_2_1_25_1","volume-title":"Search-o1: Agentic search-enhanced large reasoning models. arXiv preprint arXiv:2501.05366","author":"Li Xiaoxi","year":"2025","unstructured":"Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. 2025a. Search-o1: Agentic search-enhanced large reasoning models. arXiv preprint arXiv:2501.05366 (2025)."},{"key":"e_1_3_2_1_26_1","volume-title":"Webthinker: Empowering large reasoning models with deep research capability. arXiv preprint arXiv:2504.21776","author":"Li Xiaoxi","year":"2025","unstructured":"Xiaoxi Li, Jiajie Jin, Guanting Dong, Hongjin Qian, Yongkang Wu, Ji-Rong Wen, Yutao Zhu, and Zhicheng Dou. 2025b. Webthinker: Empowering large reasoning models with deep research capability. arXiv preprint arXiv:2504.21776 (2025)."},{"key":"e_1_3_2_1_27_1","volume-title":"The Twelfth International Conference on Learning Representations.","author":"Lightman Hunter","year":"2023","unstructured":"Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let's verify step by step. In The Twelfth International Conference on Learning Representations."},{"key":"e_1_3_2_1_28_1","volume-title":"ARPO: End-to-End Policy Optimization for GUI Agents with Experience Replay. arXiv preprint arXiv:2505.16282","author":"Lu Fanbin","year":"2025","unstructured":"Fanbin Lu, Zhisheng Zhong, Shu Liu, Chi-Wing Fu, and Jiaya Jia. 2025. ARPO: End-to-End Policy Optimization for GUI Agents with Experience Replay. arXiv preprint arXiv:2505.16282 (2025)."},{"key":"e_1_3_2_1_29_1","volume-title":"5: Advancing Math Reasoning with Offline Reinforcement Learning. arXiv preprint arXiv:2601.14716","author":"Lu Yao","year":"2026","unstructured":"Yao Lu, Dengdong Fan, Jianzheng Nie, Fan Xu, Jie Chen, Bin Zhou, and Yonghong Tian. 2026. PCL-Reasoner-V1. 5: Advancing Math Reasoning with Offline Reinforcement Learning. arXiv preprint arXiv:2601.14716 (2026)."},{"key":"e_1_3_2_1_30_1","volume-title":"GlobalRAG: Enhancing Global Reasoning in Multi-hop Question Answering via Reinforcement Learning. arXiv preprint arXiv:2510.20548","author":"Luo Jinchang","year":"2025","unstructured":"Jinchang Luo, Mingquan Cheng, Fan Wan, Ni Li, Xiaoling Xia, Shuangshuang Tian, Tingcheng Bian, Haiwei Wang, Haohuan Fu, and Yan Tao. 2025. GlobalRAG: Enhancing Global Reasoning in Multi-hop Question Answering via Reinforcement Learning. arXiv preprint arXiv:2510.20548 (2025)."},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"crossref","unstructured":"Xinji Mai Haotian Xu Zhong-Zhi Li Weinong Wang Jian Hu Yingying Zhang Wenqiang Zhang et al. 2025. Agent rl scaling law: Agent rl with spontaneous code execution for mathematical problem solving. arXiv preprint arXiv:2505.07773 (2025).","DOI":"10.52202\/085713-0251"},{"key":"e_1_3_2_1_32_1","volume-title":"The Twelfth International Conference on Learning Representations.","author":"Mialon Gr\u00e9goire","year":"2023","unstructured":"Gr\u00e9goire Mialon, Cl\u00e9mentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. 2023. Gaia: a benchmark for general ai assistants. In The Twelfth International Conference on Learning Representations."},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-emnlp.378"},{"key":"e_1_3_2_1_34_1","volume-title":"Qi He, Hongru Wang, Xiusi Chen, Dilek Hakkani-T\u00fcr, Gokhan Tur, and Heng Ji.","author":"Qian Cheng","year":"2025","unstructured":"Cheng Qian, Emre Can Acikgoz, Qi He, Hongru Wang, Xiusi Chen, Dilek Hakkani-T\u00fcr, Gokhan Tur, and Heng Ji. 2025. Toolrl: Reward is all tool learning needs. arXiv preprint arXiv:2504.13958 (2025)."},{"key":"e_1_3_2_1_35_1","unstructured":"Yulei Qin Xiaoyu Tan Zhengbao He Gang Li Haojia Lin Zongyi Li Zihan Xu Yuchen Shi Siqi Cai Renting Rui et al. 2025. Learn the ropes then trust the wins: self-imitation with progressive exploration for agentic reinforcement learning. arXiv preprint arXiv:2509.22601 (2025)."},{"key":"e_1_3_2_1_36_1","unstructured":"Qwen: An Yang Baosong Yang Beichen Zhang Binyuan Hui Bo Zheng Bowen Yu Chengyuan Li Dayiheng Liu Fei Huang Haoran Wei Huan Lin Jian Yang Jianhong Tu Jianwei Zhang Jianxin Yang Jiaxi Yang Jingren Zhou Junyang Lin Kai Dang Keming Lu Keqin Bao Kexin Yang Le Yu Mei Li Mingfeng Xue Pei Zhang Qin Zhu Rui Men Runji Lin Tianhao Li Tianyi Tang Tingyu Xia Xingzhang Ren Xuancheng Ren Yang Fan Yang Su Yichang Zhang Yu Wan Yuqiong Liu Zeyu Cui Zhenru Zhang and Zihan Qiu. 2025. Qwen2.5 Technical Report. arXiv:2412.15115 [cs.CL] https:\/\/arxiv.org\/abs\/2412.15115"},{"key":"e_1_3_2_1_37_1","volume-title":"Proceedings of the 32nd International Conference on Machine Learning (Proceedings of Machine Learning Research","volume":"1897","author":"Schulman John","year":"2015","unstructured":"John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. 2015. Trust Region Policy Optimization. In Proceedings of the 32nd International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. V37), Francis Bach and David Blei (Eds.). PMLR, Lille, France, 1889-1897."},{"key":"e_1_3_2_1_38_1","volume-title":"Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300","author":"Shao Zhihong","year":"2024","unstructured":"Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al., 2024. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 (2024)."},{"key":"e_1_3_2_1_39_1","volume-title":"HybridFlow: A Flexible and Efficient RLHF Framework. arXiv preprint arXiv: 2409.19256","author":"Sheng Guangming","year":"2024","unstructured":"Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2024. HybridFlow: A Flexible and Efficient RLHF Framework. arXiv preprint arXiv: 2409.19256 (2024)."},{"key":"e_1_3_2_1_40_1","volume-title":"Agentic reasoning and tool integration for llms via reinforcement learning. arXiv preprint arXiv:2505.01441","author":"Singh Joykirat","year":"2025","unstructured":"Joykirat Singh, Raghav Magazine, Yash Pandya, and Akshay Nambi. 2025. Agentic reasoning and tool integration for llms via reinforcement learning. arXiv preprint arXiv:2505.01441 (2025)."},{"key":"e_1_3_2_1_41_1","volume-title":"Klear-reasoner: Advancing reasoning capability via gradient-preserving clipping policy optimization. arXiv preprint arXiv:2508.07629","author":"Su Zhenpeng","year":"2025","unstructured":"Zhenpeng Su, Leiyu Pan, Xue Bai, Dening Liu, Guanting Dong, Jiaming Huang, Wenping Hu, Fuzheng Zhang, Kun Gai, and Guorui Zhou. 2025. Klear-reasoner: Advancing reasoning capability via gradient-preserving clipping policy optimization. arXiv preprint arXiv:2508.07629 (2025)."},{"key":"e_1_3_2_1_42_1","volume-title":"Zerosearch: Incentivize the search capability of llms without searching. arXiv preprint arXiv:2505.04588","author":"Sun Hao","year":"2025","unstructured":"Hao Sun, Zile Qiao, Jiayan Guo, Xuanbo Fan, Yingyan Hou, Yong Jiang, Pengjun Xie, Yan Zhang, Fei Huang, and Jingren Zhou. 2025a. Zerosearch: Incentivize the search capability of llms without searching. arXiv preprint arXiv:2505.04588 (2025)."},{"key":"e_1_3_2_1_43_1","volume-title":"Training proactive and personalized llm agents. arXiv preprint arXiv:2511.02208","author":"Sun Weiwei","year":"2025","unstructured":"Weiwei Sun, Xuhui Zhou, Weihua Du, Xingyao Wang, Sean Welleck, Graham Neubig, Maarten Sap, and Yiming Yang. 2025c. Training proactive and personalized llm agents. arXiv preprint arXiv:2511.02208 (2025)."},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.52202\/085713-5770"},{"key":"e_1_3_2_1_45_1","volume-title":"Advances in Neural Information Processing Systems","author":"Sutton Richard S","unstructured":"Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. 1999. Policy Gradient Methods for Reinforcement Learning with Function Approximation. In Advances in Neural Information Processing Systems, S. Solla, T. Leen, and K. M\u00fcller (Eds.), Vol. V12. MIT Press."},{"key":"e_1_3_2_1_46_1","unstructured":"Kimi Team Angang Du Bofei Gao Bowei Xing Changjiu Jiang Cheng Chen Cheng Li Chenjun Xiao Chenzhuang Du Chonghua Liao et al. 2025. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599 (2025)."},{"key":"e_1_3_2_1_47_1","volume-title":"V10","author":"Trivedi Harsh","year":"2022","unstructured":"Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. MuSiQue: Multihop Questions via Single-hop Question Composition. Transactions of the Association for Computational Linguistics, V10 (2022), 539-554."},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"crossref","unstructured":"Shenzhi Wang Le Yu Chang Gao Chujie Zheng Shixuan Liu Rui Lu Kai Dang Xionghui Chen Jianxin Yang Zhenru Zhang et al. 2025b. Beyond the 80\/20 rule: High-entropy minority tokens drive effective reinforcement learning for llm reasoning. arXiv preprint arXiv:2506.01939 (2025).","DOI":"10.52202\/085713-3850"},{"key":"e_1_3_2_1_49_1","unstructured":"Wenhui Wang Furu Wei Li Dong Hangbo Bao Nan Yang and Ming Zhou. 2020. MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers. arXiv:2002.10957 [cs.CL] https:\/\/arxiv.org\/abs\/2002.10957"},{"key":"e_1_3_2_1_50_1","doi-asserted-by":"crossref","unstructured":"Yiping Wang Qing Yang Zhiyuan Zeng Liliang Ren Liyuan Liu Baolin Peng Hao Cheng Xuehai He Kuan Wang Jianfeng Gao et al. 2025a. Reinforcement learning for reasoning in large language models with one training example. arXiv preprint arXiv:2504.20571 (2025).","DOI":"10.52202\/085713-4093"},{"key":"e_1_3_2_1_51_1","volume-title":"Alex Tachard Passos, William Fedus, and Amelia Glaese.","author":"Wei Jason","year":"2025","unstructured":"Jason Wei, Zhiqing Sun, Spencer Papay, Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung, Alex Tachard Passos, William Fedus, and Amelia Glaese. 2025. Browsecomp: A simple yet challenging benchmark for browsing agents. arXiv preprint arXiv:2504.12516 (2025)."},{"key":"e_1_3_2_1_52_1","volume-title":"V37","author":"Wen Muning","year":"2024","unstructured":"Muning Wen, Ziyu Wan, Jun Wang, Weinan Zhang, and Ying Wen. 2024. Reinforcing llm agents via policy optimization with action decomposition. Advances in Neural Information Processing Systems, V37 (2024), 103774-103805."},{"key":"e_1_3_2_1_53_1","volume-title":"Webwalker: Benchmarking llms in web traversal. arXiv preprint arXiv:2501.07572","author":"Wu Jialong","year":"2025","unstructured":"Jialong Wu, Wenbiao Yin, Yong Jiang, Zhenglin Wang, Zekun Xi, Runnan Fang, Linhai Zhang, Yulan He, Deyu Zhou, Pengjun Xie, et al., 2025b. Webwalker: Benchmarking llms in web traversal. arXiv preprint arXiv:2501.07572 (2025)."},{"key":"e_1_3_2_1_54_1","volume-title":"Resum: Unlocking long-horizon search intelligence via context summarization. arXiv preprint arXiv:2509.13313","author":"Wu Xixi","year":"2025","unstructured":"Xixi Wu, Kuan Li, Yida Zhao, Liwen Zhang, Litu Ou, Huifeng Yin, Zhongwang Zhang, Xinmiao Yu, Dingchu Zhang, Yong Jiang, et al., 2025a. Resum: Unlocking long-horizon search intelligence via context summarization. arXiv preprint arXiv:2509.13313 (2025)."},{"key":"e_1_3_2_1_55_1","volume-title":"Simpletir: End-to-end reinforcement learning for multi-turn tool-integrated reasoning. arXiv preprint arXiv:2509.02479","author":"Xue Zhenghai","year":"2025","unstructured":"Zhenghai Xue, Longtao Zheng, Qian Liu, Yingru Li, Xiaosen Zheng, Zejun Ma, and Bo An. 2025. Simpletir: End-to-end reinforcement learning for multi-turn tool-integrated reasoning. arXiv preprint arXiv:2509.02479 (2025)."},{"key":"e_1_3_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.52202\/085713-3904"},{"key":"e_1_3_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1259"},{"key":"e_1_3_2_1_58_1","volume-title":"Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476","author":"Yu Qiying","year":"2025","unstructured":"Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al., 2025. Dapo: An open-source llm reinforcement learning system at scale. arXiv preprint arXiv:2503.14476 (2025)."},{"key":"e_1_3_2_1_59_1","volume-title":"Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? arXiv preprint arXiv:2504.13837","author":"Yue Yang","year":"2025","unstructured":"Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang. 2025. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? arXiv preprint arXiv:2504.13837 (2025)."},{"key":"e_1_3_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.52202\/085713-1761"},{"key":"e_1_3_2_1_61_1","volume-title":"On-policy rl meets off-policy experts: Harmonizing supervised fine-tuning and reinforcement learning via dynamic weighting. arXiv preprint arXiv:2508.11408","author":"Zhang Wenhao","year":"2025","unstructured":"Wenhao Zhang, Yuexiang Xie, Yuchang Sun, Yanxi Chen, Guoyin Wang, Yaliang Li, Bolin Ding, and Jingren Zhou. 2025a. On-policy rl meets off-policy experts: Harmonizing supervised fine-tuning and reinforcement learning via dynamic weighting. arXiv preprint arXiv:2508.11408 (2025)."},{"key":"e_1_3_2_1_62_1","volume-title":"Echo chamber: Rl post-training amplifies behaviors learned in pretraining. arXiv preprint arXiv:2504.07912","author":"Zhao Rosie","year":"2025","unstructured":"Rosie Zhao, Alexandru Meterez, Sham Kakade, Cengiz Pehlevan, Samy Jelassi, and Eran Malach. 2025. Echo chamber: Rl post-training amplifies behaviors learned in pretraining. arXiv preprint arXiv:2504.07912 (2025)."},{"key":"e_1_3_2_1_63_1","unstructured":"Tianyu Zheng Tianshun Xing Qingshui Gu Taoran Liang Xingwei Qu Xin Zhou Yizhi Li Zhoufutu Wen Chenghua Lin Wenhao Huang et al. 2025. First return entropy-eliciting explore. arXiv preprint arXiv:2507.07017 (2025)."},{"key":"e_1_3_2_1_64_1","volume-title":"V44, 1","author":"Zhu Yutao","year":"2025","unstructured":"Yutao Zhu, Huaying Yuan, Shuting Wang, Jiongnan Liu, Wenhan Liu, Chenlong Deng, Haonan Chen, Zheng Liu, Zhicheng Dou, and Ji-Rong Wen. 2025. Large language models for information retrieval: A survey. ACM Transactions on Information Systems, V44, 1 (2025), 1-54."}],"event":{"name":"KDD '26: The 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining","location":"Jeju Island Republic of Korea","sponsor":["SIGMOD ACM Special Interest Group on Management of Data","SIGKDD ACM Special Interest Group on Knowledge Discovery in Data"]},"container-title":["Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/abs\/10.1145\/3770855.3817803","content-type":"text\/html","content-version":"vor","intended-application":"syndication"}],"deposited":{"date-parts":[[2026,8,7]],"date-time":"2026-08-07T01:31:43Z","timestamp":1786066303000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3770855.3817803"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,8]]},"references-count":64,"alternative-id":["10.1145\/3770855.3817803","10.1145\/3770855"],"URL":"https:\/\/doi.org\/10.1145\/3770855.3817803","relation":{},"subject":[],"published":{"date-parts":[[2026,8,8]]},"assertion":[{"value":"2026-08-08","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}