{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T10:43:07Z","timestamp":1776076987761,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":55,"publisher":"ACM","license":[{"start":{"date-parts":[[2025,4,22]],"date-time":"2025-04-22T00:00:00Z","timestamp":1745280000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,4,22]]},"DOI":"10.1145\/3696410.3714728","type":"proceedings-article","created":{"date-parts":[[2025,4,22]],"date-time":"2025-04-22T22:52:18Z","timestamp":1745362338000},"page":"896-909","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Provably Robust Federated Reinforcement Learning"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1365-3911","authenticated-orcid":false,"given":"Minghong","family":"Fang","sequence":"first","affiliation":[{"name":"University of Louisville, Louisville, KY, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1324-0225","authenticated-orcid":false,"given":"Xilong","family":"Wang","sequence":"additional","affiliation":[{"name":"Duke University, Durham, NC, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9900-9309","authenticated-orcid":false,"given":"Neil Zhenqiang","family":"Gong","sequence":"additional","affiliation":[{"name":"Duke University, Durham, NC, USA"}]}],"member":"320","published-online":{"date-parts":[[2025,4,22]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"AISTATS","author":"Bagdasaryan Eugene","year":"2020","unstructured":"Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, and Vitaly Shmatikov. How to backdoor federated learning. In AISTATS, 2020."},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSMC.1983.6313077"},{"key":"e_1_3_2_1_3_1","volume-title":"NeurIPS","author":"Baruch Gilad","year":"2019","unstructured":"Gilad Baruch, Moran Baruch, and Yoav Goldberg. A little is enough: Circumventing defenses for distributed learning. In NeurIPS, 2019."},{"key":"e_1_3_2_1_4_1","volume-title":"NeurIPS","author":"Blanchard Peva","year":"2017","unstructured":"Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, and Julien Stainer. Machine learning with adversaries: Byzantine tolerant gradient descent. In NeurIPS, 2017."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-14435-6_7"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-37456-2_14"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2021.24434"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i8.16849"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3154503"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897518.2897647"},{"key":"e_1_3_2_1_11_1","volume-title":"ICML","author":"Duan Yan","year":"2016","unstructured":"Yan Duan, Xi Chen, Rein Houthooft, John Schulman, and Pieter Abbeel. Benchmarking deep reinforcement learning for continuous control. In ICML, 2016."},{"key":"e_1_3_2_1_12_1","volume-title":"Machine Learning","author":"Dulac-Arnold Gabriel","year":"2021","unstructured":"Gabriel Dulac-Arnold, Nir Levine, Daniel J Mankowitz, Jerry Li, Cosmin Paduraru, Sven Gowal, and Todd Hester. Challenges of real-world reinforcement learning: definitions, benchmarks and analysis. In Machine Learning, 2021."},{"key":"e_1_3_2_1_13_1","volume-title":"NeurIPS","author":"Fan Xiaofeng","year":"2021","unstructured":"Xiaofeng Fan, Yining Ma, Zhongxiang Dai, Wei Jing, Cheston Tan, and Bryan Kian Hsiang Low. Fault-tolerant federated reinforcement learning with theoretical guarantee. In NeurIPS, 2021."},{"key":"e_1_3_2_1_14_1","volume-title":"USENIX Security Symposium","author":"Fang Minghong","year":"2020","unstructured":"Minghong Fang, Xiaoyu Cao, Jinyuan Jia, and Neil Gong. Local model poisoning attacks to byzantine-robust federated learning. In USENIX Security Symposium, 2020."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3564625.3567991"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2025.241796"},{"key":"e_1_3_2_1_17_1","volume-title":"CCS","author":"Fang Minghong","year":"2024","unstructured":"Minghong Fang, Zifan Zhang, Prashant Khanduri, Jia Liu, Songtao Lu, Yuchen Liu, Neil Gong, et al. Byzantine-robust decentralized federated learning. In CCS, 2024."},{"key":"e_1_3_2_1_18_1","volume-title":"WiOpt","author":"Fang Minghong","year":"2024","unstructured":"Minghong Fang, Zifan Zhang, Alvaro Velasquez, Jia Liu, et al. On the hardness of decentralized multi-agent policy evaluation under byzantine attacks. In WiOpt, 2024."},{"key":"e_1_3_2_1_19_1","volume-title":"Scientific Reports","author":"Gao Yunfei","year":"2024","unstructured":"Yunfei Gao, Mingliu Liu, Xiaopeng Yuan, Yulin Hu, Peng Sun, and Anke Schmeink. Federated deep reinforcement learning based trajectory design for uav-assisted networks with mobile ground devices. In Scientific Reports, 2024."},{"key":"e_1_3_2_1_20_1","volume-title":"AISTATS","author":"Jin Hao","year":"2022","unstructured":"Hao Jin, Yang Peng, Wenhao Yang, Shusen Wang, and Zhihua Zhang. Federated reinforcement learning with environment heterogeneity. In AISTATS, 2022."},{"key":"e_1_3_2_1_21_1","volume-title":"ICML","author":"Khodadadian Sajad","year":"2022","unstructured":"Sajad Khodadadian, Pranay Sharma, Gauri Joshi, and Siva Theja Maguluri. Federated reinforcement learning: Linear speedup under markovian sampling. In ICML, 2022."},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1177\/0278364913495721"},{"key":"e_1_3_2_1_23_1","volume-title":"AISTATS","author":"Lei Lihua","year":"2017","unstructured":"Lihua Lei and Michael Jordan. Less than a single pass: Stochastically controlled stochastic gradient. In AISTATS, 2017."},{"key":"e_1_3_2_1_24_1","volume-title":"Federated and Transfer Learning","author":"Liang Xinle","year":"2022","unstructured":"Xinle Liang, Yang Liu, Tianjian Chen, Ming Liu, and Qiang Yang. Federated transfer reinforcement learning for autonomous driving. In Federated and Transfer Learning, 2022."},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/SPW50608.2020.00027"},{"key":"e_1_3_2_1_26_1","volume":"201","author":"Liu Boyi","unstructured":"Boyi Liu, Lujia Wang, and Ming Liu. Lifelong federated reinforcement learning: a learning architecture for navigation in cloud robotic systems. In IEEE Robotics and Automation Letters, 2019.","journal-title":"Automation Letters"},{"key":"e_1_3_2_1_27_1","volume-title":"Kee Yuan Ngiam, Leo Anthony Celi, Xingzhi Sun, and Mengling Feng. Reinforcement learning for clinical decision support in critical care: comprehensive review. In Journal of medical Internet research","author":"Liu Siqi","year":"2020","unstructured":"Siqi Liu, Kay Choong See, Kee Yuan Ngiam, Leo Anthony Celi, Xingzhi Sun, and Mengling Feng. Reinforcement learning for clinical decision support in critical care: comprehensive review. In Journal of medical Internet research, 2020."},{"key":"e_1_3_2_1_28_1","volume-title":"Local environment poisoning attacks on federated reinforcement learning. arXiv preprint arXiv:2303.02725","author":"Ma Evelyn","year":"2023","unstructured":"Evelyn Ma, Praneet Rathi, and S Rasoul Etesami. Local environment poisoning attacks on federated reinforcement learning. arXiv preprint arXiv:2303.02725, 2023."},{"key":"e_1_3_2_1_29_1","volume-title":"AISTATS","author":"McMahan H Brendan","year":"2017","unstructured":"H Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, et al. Communication-efficient learning of deep networks from decentralized data. In AISTATS, 2017."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.3150\/14-BEJ645"},{"key":"e_1_3_2_1_31_1","volume-title":"ICML","author":"Mnih Volodymyr","year":"2016","unstructured":"Volodymyr Mnih, Adria Puigdomenech Badia, Mehdi Mirza, Alex Graves, Timothy Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu. Asynchronous methods for deep reinforcement learning. In ICML, 2016."},{"key":"e_1_3_2_1_32_1","volume-title":"USENIX Security Symposium","author":"Mozaffari Hamid","year":"2023","unstructured":"Hamid Mozaffari, Virat Shejwalkar, and Amir Houmansadr. Every vote counts: Ranking-based training of federated learning to resist poisoning attacks. In USENIX Security Symposium, 2023."},{"key":"e_1_3_2_1_33_1","volume-title":"Vedavyas Panneershelvam, Mustafa Suleyman, Charles Beattie, Stig Petersen, et al. Massively parallel methods for deep reinforcement learning. arXiv preprint arXiv:1507.04296","author":"Nair Arun","year":"2015","unstructured":"Arun Nair, Praveen Srinivasan, Sam Blackwell, Cagdas Alcicek, Rory Fearon, Alessandro De Maria, Vedavyas Panneershelvam, Mustafa Suleyman, Charles Beattie, Stig Petersen, et al. Massively parallel methods for deep reinforcement learning. arXiv preprint arXiv:1507.04296, 2015."},{"key":"e_1_3_2_1_34_1","volume-title":"USENIX Security Symposium","author":"Nguyen Thien Duc","year":"2022","unstructured":"Thien Duc Nguyen, Phillip Rieger, Roberta De Viti, Huili Chen, Bj\u00f6rn B Brandenburg, Hossein Yalame, Helen M\u00f6llering, Hossein Fereidooni, Samuel Marchal, Markus Miettinen, et al. Flame: Taming backdoors in federated learning. In USENIX Security Symposium, 2022."},{"key":"e_1_3_2_1_35_1","volume-title":"USENIX Security Symposium","author":"Pan Xudong","year":"2020","unstructured":"Xudong Pan, Mi Zhang, Duocai Wu, Qifan Xiao, Shouling Ji, and Min Yang. Justinian's gaavernor: Robust distributed learning with gradient aggregation agent. In USENIX Security Symposium, 2020."},{"key":"e_1_3_2_1_36_1","volume-title":"NeurIPS","author":"Rajput Shashank","year":"2019","unstructured":"Shashank Rajput, Hongyi Wang, Zachary Charles, and Dimitris Papailiopoulos. Detox: A redundancy-based framework for faster and more robust gradient aggregation. In NeurIPS, 2019."},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2022.23156"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2021.24498"},{"key":"e_1_3_2_1_39_1","volume-title":"Reinforcement learning: An introduction","author":"Sutton Richard S","year":"2018","unstructured":"Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018."},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.5555\/3091529.3091572"},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2012.6386109"},{"key":"e_1_3_2_1_42_1","volume-title":"Nature","author":"Vinyals Oriol","year":"2019","unstructured":"Oriol Vinyals, Igor Babuschkin, Wojciech M Czarnecki, Micha\u00ebl Mathieu, Andrew Dudzik, Junyoung Chung, David H Choi, Richard Powell, Timo Ewalds, Petko Georgiev, et al. Grandmaster level in starcraft ii using multi-agent reinforcement learning. In Nature, 2019."},{"key":"e_1_3_2_1_43_1","volume-title":"NeurIPS","author":"Wang Hongyi","year":"2020","unstructured":"Hongyi Wang, Kartik Sreenivasan, Shashank Rajput, Harit Vishwakarma, Saurabh Agarwal, Jy-yong Sohn, Kangwook Lee, and Dimitris Papailiopoulos. Attack of the tails: Yes, you really can backdoor federated learning. In NeurIPS, 2020."},{"key":"e_1_3_2_1_44_1","volume":"202","author":"Wang Xiaofei","unstructured":"Xiaofei Wang, Chenyang Wang, Xiuhua Li, Victor CM Leung, and Tarik Taleb. Federated deep reinforcement learning for internet of things with decentralized cooperative edge caching. In IEEE Internet of Things Journal, 2020.","journal-title":"Things Journal"},{"key":"e_1_3_2_1_45_1","volume-title":"Machine learning","author":"Williams Ronald J","year":"1992","unstructured":"Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. In Machine learning, 1992."},{"key":"e_1_3_2_1_46_1","volume-title":"ICLR","author":"Xie Chulin","year":"2020","unstructured":"Chulin Xie, Keli Huang, Pin-Yu Chen, and Bo Li. Dba: Distributed backdoor attacks against federated learning. In ICLR, 2020."},{"key":"e_1_3_2_1_47_1","volume-title":"ICML","author":"Xie Cong","year":"2019","unstructured":"Cong Xie, Sanmi Koyejo, and Indranil Gupta. Zeno: Distributed stochastic gradient descent with suspicion-based fault-tolerance. In ICML, 2019."},{"key":"e_1_3_2_1_48_1","volume-title":"ICML","author":"Xie Yueqi","year":"2024","unstructured":"Yueqi Xie, Minghong Fang, and Neil Zhenqiang Gong. Fedredefense: Defending against model poisoning attacks for federated learning using model update reconstruction error. In ICML, 2024."},{"key":"e_1_3_2_1_49_1","volume-title":"ICML","author":"Yin Dong","year":"2018","unstructured":"Dong Yin, Yudong Chen, Ramchandran Kannan, and Peter Bartlett. Byzantine-robust distributed learning: Towards optimal statistical rates. In ICML, 2018."},{"key":"e_1_3_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3589334.3645492"},{"key":"e_1_3_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.23919\/ACC55779.2023.10156236"},{"key":"e_1_3_2_1_52_1","volume-title":"ICML","author":"Zhang Kaiqing","year":"2018","unstructured":"Kaiqing Zhang, Zhuoran Yang, Han Liu, Tong Zhang, and Tamer Basar. Fully decentralized multi-agent reinforcement learning with networked agents. In ICML, 2018."},{"key":"e_1_3_2_1_53_1","volume-title":"ICML","author":"Zhang Xuezhou","year":"2020","unstructured":"Xuezhou Zhang, Yuzhe Ma, Adish Singla, and Xiaojin Zhu. Adaptive reward-poisoning attacks against reinforcement learning. In ICML, 2020."},{"key":"e_1_3_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3534678.3539231"},{"key":"e_1_3_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.23919\/IFIPNetworking62109.2024.10619763"}],"event":{"name":"WWW '25: The ACM Web Conference 2025","location":"Sydney NSW Australia","acronym":"WWW '25","sponsor":["SIGWEB ACM Special Interest Group on Hypertext, Hypermedia, and Web"]},"container-title":["Proceedings of the ACM on Web Conference 2025"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3696410.3714728","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3696410.3714728","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:18:57Z","timestamp":1750295937000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3696410.3714728"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,22]]},"references-count":55,"alternative-id":["10.1145\/3696410.3714728","10.1145\/3696410"],"URL":"https:\/\/doi.org\/10.1145\/3696410.3714728","relation":{},"subject":[],"published":{"date-parts":[[2025,4,22]]},"assertion":[{"value":"2025-04-22","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}