{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,12]],"date-time":"2026-03-12T15:52:55Z","timestamp":1773330775080,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":69,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,9,18]],"date-time":"2020-09-18T00:00:00Z","timestamp":1600387200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"the Innovation Research Group Project of NSFC","award":["61921003, 61772084, 61720106007, 61832010"],"award-info":[{"award-number":["61921003, 61772084, 61720106007, 61832010"]}]},{"name":"the Fundamental Research Funds for the Central Universities","award":["2019XD-A13"],"award-info":[{"award-number":["2019XD-A13"]}]},{"name":"the 111 Project","award":["B18008"],"award-info":[{"award-number":["B18008"]}]},{"name":"Alibaba Innovation Research Program"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,9,21]]},"DOI":"10.1145\/3372224.3419186","type":"proceedings-article","created":{"date-parts":[[2020,9,19]],"date-time":"2020-09-19T02:16:25Z","timestamp":1600481785000},"page":"1-14","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":100,"title":["OnRL"],"prefix":"10.1145","author":[{"given":"Huanhuan","family":"Zhang","sequence":"first","affiliation":[{"name":"Beijing Univ"}]},{"given":"Anfu","family":"Zhou","sequence":"additional","affiliation":[{"name":"Beijing Univ"}]},{"given":"Jiamin","family":"Lu","sequence":"additional","affiliation":[{"name":"Beijing Univ"}]},{"given":"Ruoxuan","family":"Ma","sequence":"additional","affiliation":[{"name":"Beijing Univ"}]},{"given":"Yuhan","family":"Hu","sequence":"additional","affiliation":[{"name":"Beijing Univ"}]},{"given":"Cong","family":"Li","sequence":"additional","affiliation":[{"name":"Beijing Univ"}]},{"given":"Xinyu","family":"Zhang","sequence":"additional","affiliation":[{"name":"University of California San Diego"}]},{"given":"Huadong","family":"Ma","sequence":"additional","affiliation":[{"name":"Beijing Univ"}]},{"given":"Xiaojiang","family":"Chen","sequence":"additional","affiliation":[{"name":"Taobao Inc."}]}],"member":"320","published-online":{"date-parts":[[2020,9,18]]},"reference":[{"key":"e_1_3_2_1_1_1","unstructured":"2017. Closing the Simulation-to-Reality Gap for Deep Robotic Learning. https:\/\/ai.googleblog.com\/2017\/10\/closing-simulation-to-reality-gap-for.html.  2017. Closing the Simulation-to-Reality Gap for Deep Robotic Learning. https:\/\/ai.googleblog.com\/2017\/10\/closing-simulation-to-reality-gap-for.html."},{"key":"e_1_3_2_1_2_1","unstructured":"2017. Video Quality of Service (QOS) Tutorial. https:\/\/www.cisco.com\/c\/en\/us\/support\/docs\/quality-of-service-qos\/qos-video\/212134-Video-Quality-of-Service-QOS-Tutorial.html.  2017. Video Quality of Service (QOS) Tutorial. https:\/\/www.cisco.com\/c\/en\/us\/support\/docs\/quality-of-service-qos\/qos-video\/212134-Video-Quality-of-Service-QOS-Tutorial.html."},{"key":"e_1_3_2_1_3_1","unstructured":"2018. WebRTC Homepage. https:\/\/webrtc.org\/.  2018. WebRTC Homepage. https:\/\/webrtc.org\/."},{"key":"e_1_3_2_1_4_1","unstructured":"2019. Alibaba Cloud Overview. https:\/\/www.assistanz.com\/alibaba-cloud-overview\/.  2019. Alibaba Cloud Overview. https:\/\/www.assistanz.com\/alibaba-cloud-overview\/."},{"key":"e_1_3_2_1_5_1","unstructured":"2019. Cisco Visual Networking Index: Forecast and Trends. https:\/\/www.cisco.com\/c\/en\/us\/solutions\/collateral\/service-provider\/visual-networking-index-vni\/white-paper-c11-741490.html?dtid=osscdc000283.  2019. Cisco Visual Networking Index: Forecast and Trends. https:\/\/www.cisco.com\/c\/en\/us\/solutions\/collateral\/service-provider\/visual-networking-index-vni\/white-paper-c11-741490.html?dtid=osscdc000283."},{"key":"e_1_3_2_1_6_1","unstructured":"2019. Taobao statistics market share trends insights. https:\/\/www.chinainternetwatch.com\/tag\/taobao\/.  2019. Taobao statistics market share trends insights. https:\/\/www.chinainternetwatch.com\/tag\/taobao\/."},{"key":"e_1_3_2_1_7_1","unstructured":"2020. 30 amazing taobao statistics and facts (2020). By the numbers. https:\/\/expandedramblings.com\/index.php\/taobao-statistics\/.  2020. 30 amazing taobao statistics and facts (2020). By the numbers. https:\/\/expandedramblings.com\/index.php\/taobao-statistics\/."},{"key":"e_1_3_2_1_8_1","unstructured":"2020. Alibaba cloud Server Load Balancer. https:\/\/www.alibabacloud.com\/help\/doc-detail\/27544.htm?spm=a2c63.p38356.b99.5.4bc42299yeUcWm.  2020. Alibaba cloud Server Load Balancer. https:\/\/www.alibabacloud.com\/help\/doc-detail\/27544.htm?spm=a2c63.p38356.b99.5.4bc42299yeUcWm."},{"key":"e_1_3_2_1_9_1","unstructured":"2020. Chromium in webrtc. https:\/\/chromium.googlesource.com\/external\/webrtc\/.  2020. Chromium in webrtc. https:\/\/chromium.googlesource.com\/external\/webrtc\/."},{"key":"e_1_3_2_1_10_1","unstructured":"2020. Core ML framework. https:\/\/developer.apple.com\/documentation\/coreml.  2020. Core ML framework. https:\/\/developer.apple.com\/documentation\/coreml."},{"key":"e_1_3_2_1_11_1","unstructured":"2020. Linux Traffic Control. https:\/\/events.static.linuxfound.org\/sites\/events\/files\/slides\/Linux_traffic_control.pdf.  2020. Linux Traffic Control. https:\/\/events.static.linuxfound.org\/sites\/events\/files\/slides\/Linux_traffic_control.pdf."},{"key":"e_1_3_2_1_12_1","unstructured":"2020. Tensorflow Lite. https:\/\/www.tensorflow.org\/lite.  2020. Tensorflow Lite. https:\/\/www.tensorflow.org\/lite."},{"key":"e_1_3_2_1_13_1","unstructured":"2020. Tensorflow source code. https:\/\/github.com\/tensorflow\/tensorflow\/tree\/master\/tensorflow\/tools..  2020. Tensorflow source code. https:\/\/github.com\/tensorflow\/tensorflow\/tree\/master\/tensorflow\/tools.."},{"key":"e_1_3_2_1_14_1","unstructured":"2020. TFLearn: Deep learning library featuring a higher-level API for TensorFlow. http:\/\/tflearn.org\/.  2020. TFLearn: Deep learning library featuring a higher-level API for TensorFlow. http:\/\/tflearn.org\/."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.image.2011.10.003"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11797"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3323679.3326523"},{"key":"e_1_3_2_1_18_1","volume-title":"NIPS 2003","author":"Bottou L\u00e9on","year":"2003"},{"key":"e_1_3_2_1_19_1","volume-title":"Proceedings of the 31th International Conference on Machine Learning, ICML 2014","author":"Bou-Ammar Haitham","year":"2014"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2018.8460875"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3012426.3022184"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNET.2017.2703615"},{"key":"e_1_3_2_1_23_1","first-page":"519","article-title":"Live video broadcasting mobile application for social sharing","volume":"14","author":"Clements Jason","year":"2016","journal-title":"US Patent App."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"crossref","unstructured":"Yann Le Cun Ido Kanter and Sara A. Solla. [n.d.]. Eigenvalues of covariance matrices: Application to neural-network learning. Physical Review Letters 66 18 ([n. d.]) 2396--2399.  Yann Le Cun Ido Kanter and Sara A. Solla. [n.d.]. Eigenvalues of covariance matrices: Application to neural-network learning. Physical Review Letters 66 18 ([n. d.]) 2396--2399.","DOI":"10.1103\/PhysRevLett.66.2396"},{"key":"e_1_3_2_1_25_1","volume-title":"PCC: Re-architecting Congestion Control for Consistent High Performance. In 12th USENIX Symposium on Networked Systems Design and Implementation (NSDI 15)","author":"Dong Mo","year":"2015"},{"key":"e_1_3_2_1_26_1","volume-title":"PCC Vivace: Online-Learning Congestion Control. In 15th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2018","author":"Dong Mo","year":"2018"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/1953048.2021068"},{"key":"e_1_3_2_1_28_1","volume-title":"15th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2018","author":"Fouladi Sadjad","year":"2018"},{"key":"e_1_3_2_1_29_1","volume-title":"3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7--9, 2015, Conference Track Proceedings. http:\/\/arxiv.org\/abs\/1412","author":"Ian"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1400097.1400105"},{"key":"e_1_3_2_1_31_1","first-page":"141","article-title":"Live sports virtual reality broadcasts: Copyright and other protections","volume":"16","author":"Hopkins Marie","year":"2017","journal-title":"Duke L. & Tech. Rev."},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11235-010-9351-x"},{"key":"e_1_3_2_1_33_1","volume-title":"Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9--15","author":"Jay Nathan","year":"2019"},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2934872.2934907"},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2413176.2413189"},{"key":"e_1_3_2_1_36_1","first-page":"308","article-title":"Improvement of TCP Reno Congestion Control Protocol","volume":"163","author":"Jiang Yueqiu","year":"2014","journal-title":"Sensors & Transducers"},{"key":"e_1_3_2_1_37_1","unstructured":"Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In 3rd International Conference on Learning Representations ICLR 2015 San Diego CA USA May 7--9 2015 Conference Track Proceedings. http:\/\/arxiv.org\/abs\/1412.6980  Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In 3rd International Conference on Learning Representations ICLR 2015 San Diego CA USA May 7--9 2015 Conference Track Proceedings. http:\/\/arxiv.org\/abs\/1412.6980"},{"key":"e_1_3_2_1_38_1","volume-title":"6th International Conference on Learning Representations, ICLR","author":"Lin Yujun","year":"2018"},{"key":"e_1_3_2_1_39_1","first-page":"310","article-title":"Interactions between vehicle and teleoperations system","volume":"15","author":"Kelly Lockwood Amanda Lee","year":"2019","journal-title":"US Patent App."},{"key":"e_1_3_2_1_40_1","volume-title":"Proceedings of the 2019 Reinforcement Learning for Real Life Workshop.","author":"Mao Hongzi","year":"2019"},{"key":"e_1_3_2_1_41_1","volume-title":"Park: An Open Platform for Learning-Augmented Computer Systems. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019","author":"Mao Hongzi","year":"2019"},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3098822.3098843"},{"key":"e_1_3_2_1_43_1","volume-title":"Towards Safe Online Reinforcement Learning in Computer Systems. In 33rd Conference on Neural Information Processing Systems (NeurIPS","author":"Mao Hongzi","year":"2019"},{"key":"e_1_3_2_1_44_1","volume-title":"Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, 20--22","author":"McMahan Brendan","year":"2017"},{"key":"e_1_3_2_1_45_1","volume-title":"Proceedings of the 33nd International Conference on Machine Learning, ICML 2016","author":"Mnih Volodymyr","year":"2016"},{"key":"e_1_3_2_1_46_1","unstructured":"Volodymyr Mnih Koray Kavukcuoglu David Silver Alex Graves Ioannis Antonoglou Daan Wierstra and Martin A. Riedmiller. 2013. Playing Atari with Deep Reinforcement Learning. CoRR abs\/1312.5602 (2013). arXiv:1312.5602 http:\/\/arxiv.org\/abs\/1312.5602  Volodymyr Mnih Koray Kavukcuoglu David Silver Alex Graves Ioannis Antonoglou Daan Wierstra and Martin A. Riedmiller. 2013. Playing Atari with Deep Reinforcement Learning. CoRR abs\/1312.5602 (2013). arXiv:1312.5602 http:\/\/arxiv.org\/abs\/1312.5602"},{"key":"e_1_3_2_1_47_1","volume-title":"Deep Online Learning Via Meta-Learning: Continual Adaptation for Model-Based RL. In 7th International Conference on Learning Representations, ICLR 2019","author":"Nagabandi Anusha","year":"2019"},{"key":"e_1_3_2_1_48_1","volume-title":"Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems","author":"Pirotta Matteo","year":"2013"},{"key":"e_1_3_2_1_49_1","volume-title":"5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24--26, 2017, Conference Track Proceedings.","author":"Ravi Sachin","year":"2017"},{"key":"e_1_3_2_1_50_1","volume-title":"Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2011","author":"Ross St\u00e9phane","year":"2011"},{"key":"e_1_3_2_1_51_1","volume-title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018","author":"Sahoo Doyen","year":"2018"},{"key":"e_1_3_2_1_52_1","unstructured":"John Schulman Filip Wolski Prafulla Dhariwal Alec Radford and Oleg Klimov. 2017. Proximal Policy Optimization Algorithms. CoRR abs\/1707.06347 (2017).  John Schulman Filip Wolski Prafulla Dhariwal Alec Radford and Oleg Klimov. 2017. Proximal Policy Optimization Algorithms. CoRR abs\/1707.06347 (2017)."},{"key":"e_1_3_2_1_53_1","doi-asserted-by":"crossref","unstructured":"Ivan Slivar Mirko Suznjevic and Lea Skorin-Kapov. 2018. Game Categorization for Deriving QoE-Driven Video Encoding Configuration Strategies for Cloud Gaming. TOMM 14 3s (2018) 56:1--56:24.  Ivan Slivar Mirko Suznjevic and Lea Skorin-Kapov. 2018. Game Categorization for Deriving QoE-Driven Video Encoding Configuration Strategies for Cloud Gaming. TOMM 14 3s (2018) 56:1--56:24.","DOI":"10.1145\/3132041"},{"key":"e_1_3_2_1_54_1","unstructured":"Taobao. 2020. Taobao Live APP. https:\/\/apps.apple.com\/cn\/app\/E6B798E5AE9DE79BB4E692AD\/id1448831879\/.  Taobao. 2020. Taobao Live APP. https:\/\/apps.apple.com\/cn\/app\/E6B798E5AE9DE79BB4E692AD\/id1448831879\/."},{"key":"e_1_3_2_1_55_1","unstructured":"Taobao. 2020. Taobao-live monthly report 2020 February. https:\/\/mp.weixin.qq.com\/s\/wNMfYAYcTI5T3O10x6hA2w\/.  Taobao. 2020. Taobao-live monthly report 2020 February. https:\/\/mp.weixin.qq.com\/s\/wNMfYAYcTI5T3O10x6hA2w\/."},{"key":"e_1_3_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN.2009.5178586"},{"key":"e_1_3_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/3343031.3350904"},{"key":"e_1_3_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNET.2017.2765246"},{"key":"e_1_3_2_1_59_1","volume-title":"Toward Intuitive Teleoperation in Surgery: Human-Centric Evaluation of Teleoperation Algorithms for Robotic Needle Steering. In 2018 IEEE International Conference on Robotics and Automation, ICRA 2018","author":"Wang Ziheng","year":"2018"},{"key":"e_1_3_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/2486001.2486020"},{"key":"e_1_3_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.5555\/2482626.2482670"},{"key":"e_1_3_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/2789168.2790118"},{"key":"e_1_3_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/3081333.3081367"},{"key":"e_1_3_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/2462456.2464453"},{"key":"e_1_3_2_1_65_1","volume-title":"17th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2020","author":"Yan Francis Y.","year":"2020"},{"key":"e_1_3_2_1_66_1","volume-title":"2018 USENIX Annual Technical Conference, USENIX ATC 2018","author":"Yan Francis Y.","year":"2018"},{"key":"e_1_3_2_1_67_1","unstructured":"Xin Yao Tianchi Huang Chenglei Wu Rui-Xiao Zhang and Lifeng Sun. 2019. Federated Learning with Additional Mechanisms on Clients to Reduce Communication Costs. CoRR abs\/1908.05891 (2019).  Xin Yao Tianchi Huang Chenglei Wu Rui-Xiao Zhang and Lifeng Sun. 2019. Federated Learning with Additional Mechanisms on Clients to Reduce Communication Costs. CoRR abs\/1908.05891 (2019)."},{"key":"e_1_3_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/2785956.2787498"},{"key":"e_1_3_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/3300061.3345430"}],"event":{"name":"MobiCom '20: The 26th Annual International Conference on Mobile Computing and Networking","location":"London United Kingdom","acronym":"MobiCom '20","sponsor":["SIGMOBILE ACM Special Interest Group on Mobility of Systems, Users, Data and Computing"]},"container-title":["Proceedings of the 26th Annual International Conference on Mobile Computing and Networking"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3372224.3419186","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3372224.3419186","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:32:10Z","timestamp":1750195930000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3372224.3419186"}},"subtitle":["improving mobile video telephony via online reinforcement learning"],"short-title":[],"issued":{"date-parts":[[2020,9,18]]},"references-count":69,"alternative-id":["10.1145\/3372224.3419186","10.1145\/3372224"],"URL":"https:\/\/doi.org\/10.1145\/3372224.3419186","relation":{},"subject":[],"published":{"date-parts":[[2020,9,18]]},"assertion":[{"value":"2020-09-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}