{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T03:00:47Z","timestamp":1775617247050,"version":"3.50.1"},"reference-count":35,"publisher":"Association for Computing Machinery (ACM)","issue":"7","license":[{"start":{"date-parts":[[2024,5,16]],"date-time":"2024-05-16T00:00:00Z","timestamp":1715817600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62102430"],"award-info":[{"award-number":["62102430"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Hunan Young Talents","award":["2020RC3027"],"award-info":[{"award-number":["2020RC3027"]}]},{"DOI":"10.13039\/501100018579","name":"Training Program for Excellent Young Innovators of Changsha","doi-asserted-by":"crossref","award":["kq2206001"],"award-info":[{"award-number":["kq2206001"]}],"id":[{"id":"10.13039\/501100018579","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2024,7,31]]},"abstract":"<jats:p>The popularization of video streaming brings challenges in satisfying diverse Quality of Service (QoS) requirements. The multipath extension of the Quick UDP Internet Connection (QUIC) protocol, also called MPQUIC, has the potential to improve video streaming performance with multiple simultaneously transmitting paths. The multipath scheduler of MPQUIC determines how to distribute the packets onto different paths. However, while applying current multipath schedulers into MPQUIC, our experimental results show that they fail to adapt to various receive buffer sizes of different devices and comprehensive QoS requirements of video streaming. These problems are especially severe under heterogeneous and dynamic network environments. To tackle these problems, we propose MARS, a Multi-agent deep Reinforcement learning (MADRL)-based Multipath QUIC Scheduler, which is able to promptly adapt to dynamic network environments. It exploits the MADRL method to learn a neural network for each path and generate scheduling policy. Besides, it introduces a novel multi-objective reward function that takes out-of-order queue size and different QoS metrics into consideration to realize adaptive scheduling optimization. We implement MARS in an MPQUIC prototype and deploy in Dynamic Adaptive Streaming over HTTP system. Then, we compare it with the state-of-the-art multipath schedulers in both emulated and real-world networks. Experimental results show that MARS outperforms the other schedulers with better adaptive capability regarding the receive buffer sizes and QoS.<\/jats:p>","DOI":"10.1145\/3649139","type":"journal-article","created":{"date-parts":[[2024,3,15]],"date-time":"2024-03-15T12:02:31Z","timestamp":1710504151000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":14,"title":["Multi-agent DRL-based Multipath Scheduling for Video Streaming with QUIC"],"prefix":"10.1145","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-1437-763X","authenticated-orcid":false,"given":"Xueqiang","family":"Han","sequence":"first","affiliation":[{"name":"National University of Defense Technology, Changsha, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5082-5727","authenticated-orcid":false,"given":"Biao","family":"Han","sequence":"additional","affiliation":[{"name":"National University of Defense Technology, Changsha, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-4546-0631","authenticated-orcid":false,"given":"Jinrong","family":"Li","sequence":"additional","affiliation":[{"name":"National University of Defense Technology, Changsha, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7672-0915","authenticated-orcid":false,"given":"Congxi","family":"Song","sequence":"additional","affiliation":[{"name":"National University of Defense Technology, Changsha, China"}]}],"member":"320","published-online":{"date-parts":[[2024,5,16]]},"reference":[{"issue":"8","key":"e_1_3_2_2_2","doi-asserted-by":"crossref","first-page":"1885","DOI":"10.1109\/TMC.2018.2868670","article-title":"Practical and dynamic buffer sizing using LearnQueue","volume":"18","author":"Bouacida Nader","year":"2018","unstructured":"Nader Bouacida and Basem Shihada. 2018. Practical and dynamic buffer sizing using LearnQueue. IEEE Trans. Mobile Comput. 18, 8 (2018), 1885\u20131897.","journal-title":"IEEE Trans. Mobile Comput."},{"key":"e_1_3_2_3_2","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1145\/3143361.3143370","volume-title":"Proceedings of the 13th International Conference on Emerging Networking Experiments and Technologies","author":"Coninck Quentin De","year":"2017","unstructured":"Quentin De Coninck and Olivier Bonaventure. 2017. Multipath QUIC: Design and evaluation. In Proceedings of the 13th International Conference on Emerging Networking Experiments and Technologies. 160\u2013166."},{"key":"e_1_3_2_4_2","first-page":"431","volume-title":"Proceedings of the IFIP Networking Conference (IFIP Networking\u201916)","author":"Ferlin Simone","year":"2016","unstructured":"Simone Ferlin, \u00d6zg\u00fc Alay, Olivier Mehani, and Roksana Boreli. 2016. BLEST: Blocking estimation-based MPTCP scheduler for heterogeneous networks. In Proceedings of the IFIP Networking Conference (IFIP Networking\u201916). IEEE, 431\u2013439."},{"key":"e_1_3_2_5_2","first-page":"123","volume-title":"Proceedings of the IEEE 22nd International Symposium of Quality of Service (IWQoS\u201914)","author":"Ferlin-Oliveira Simone","year":"2014","unstructured":"Simone Ferlin-Oliveira, Thomas Dreibholz, and \u00d6zg\u00fc Alay. 2014. Tackling the challenge of bufferbloat in multi-path transport over heterogeneous wireless networks. In Proceedings of the IEEE 22nd International Symposium of Quality of Service (IWQoS\u201914). IEEE, 123\u2013128."},{"key":"e_1_3_2_6_2","doi-asserted-by":"crossref","DOI":"10.17487\/rfc6824","volume-title":"TCP Extensions for Multipath Operation with Multiple Addresses","author":"Ford Alan","year":"2013","unstructured":"Alan Ford, Costin Raiciu, Mark Handley, and Olivier Bonaventure. 2013. TCP Extensions for Multipath Operation with Multiple Addresses. Technical Report."},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/2999572.2999606"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/IWQoS57198.2023.10188744"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/2413176.2413206"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNSM.2021.3093302"},{"key":"e_1_3_2_11_2","volume-title":"Linux Conf Au","author":"Hemminger Stephen","year":"2005","unstructured":"Stephen Hemminger et\u00a0al. 2005. Network emulation with NetEm. In Linux Conf Au, Vol. 5. Citeseer."},{"key":"e_1_3_2_12_2","doi-asserted-by":"crossref","unstructured":"Jana Iyengar and Martin Thomson. 2021. QUIC: A UDP-based Multiplexed and Secure Transport. RFC 9000. (May 2021). Retrieved from https:\/\/datatracker.ietf.org\/doc\/rfc9000\/","DOI":"10.17487\/RFC9000"},{"key":"e_1_3_2_13_2","first-page":"1","volume-title":"Proceedings of the IEEE\/ACM 30th International Symposium on Quality of Service (IWQoS\u201922)","author":"Ji Xiaolan","year":"2022","unstructured":"Xiaolan Ji, Biao Han, Ruidong Li, Cao Xu, Yahui Li, and Jinshu Su. 2022. ACCeSS: Adaptive QoS-aware congestion control for multipath TCP. In Proceedings of the IEEE\/ACM 30th International Symposium on Quality of Service (IWQoS\u201922). IEEE, 1\u201310."},{"key":"e_1_3_2_14_2","first-page":"329","volume-title":"Proceedings of the Internet Measurement Conference","author":"Jiang Haiqing","year":"2012","unstructured":"Haiqing Jiang, Yaogong Wang, Kyunghan Lee, and Injong Rhee. 2012. Tackling bufferbloat in 3G\/4G networks. In Proceedings of the Internet Measurement Conference. 329\u2013342."},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/2413176.2413189"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNET.2013.2274462"},{"key":"e_1_3_2_17_2","first-page":"1222","volume-title":"Proceedings of the IEEE International Conference on Communications (ICC\u201914)","author":"Kuhn Nicolas","year":"2014","unstructured":"Nicolas Kuhn, Emmanuel Lochin, Ahlem Mifdaoui, Golam Sarwar, Olivier Mehani, and Roksana Boreli. 2014. DAPS: Intelligent delay-aware packet scheduling for multipath transport. In Proceedings of the IEEE International Conference on Communications (ICC\u201914). IEEE, 1222\u20131227."},{"key":"e_1_3_2_18_2","article-title":"Continuous control with deep reinforcement learning","author":"Lillicrap Timothy P.","year":"2015","unstructured":"Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2015. Continuous control with deep reinforcement learning. Retrieved from https:\/\/arXiv:1509.02971","journal-title":"Retrieved from"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3143361.3143376"},{"key":"e_1_3_2_20_2","article-title":"Multi-agent actor-critic for mixed cooperative-competitive environments","volume":"30","author":"Lowe Ryan","year":"2017","unstructured":"Ryan Lowe, Yi I. Wu, Aviv Tamar, Jean Harb, Pieter Abbeel, and Igor Mordatch. 2017. Multi-agent actor-critic for mixed cooperative-competitive environments. Adv. Neural Info. Process. Syst. 30 (2017).","journal-title":"Adv. Neural Info. Process. Syst."},{"key":"e_1_3_2_21_2","first-page":"193","volume-title":"Proceedings of the ACM SIGCOMM Conference","author":"Meng Zili","year":"2022","unstructured":"Zili Meng, Yaning Guo, Chen Sun, Bo Wang, Justine Sherry, Hongqiang Harry Liu, and Mingwei Xu. 2022. Achieving consistent low latency for wireless real-time communications with the shortest control loop. In Proceedings of the ACM SIGCOMM Conference. 193\u2013206."},{"key":"e_1_3_2_22_2","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1145\/2630088.2631977","volume-title":"Proceedings of the ACM SIGCOMM Workshop on Capacity Sharing Workshop","author":"Paasch Christoph","year":"2014","unstructured":"Christoph Paasch, Simone Ferlin, Ozgu Alay, and Olivier Bonaventure. 2014. Experimental evaluation of multipath TCP schedulers. In Proceedings of the ACM SIGCOMM Workshop on Capacity Sharing Workshop. 27\u201332."},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/2043164.2018467"},{"key":"e_1_3_2_24_2","first-page":"399","volume-title":"Proceedings of the 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201912)","author":"Raiciu Costin","year":"2012","unstructured":"Costin Raiciu, Christoph Paasch, Sebastien Barre, Alan Ford, Michio Honda, Fabien Duchene, Olivier Bonaventure, and Mark Handley. 2012. How hard can it be? Designing and implementing a deployable multipath TCP. In Proceedings of the 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201912). 399\u2013412."},{"key":"e_1_3_2_25_2","unstructured":"X. Sandvine. 2023. 2023 global internet phenomena report. Retrieved from https:\/\/www.sandvine.com"},{"key":"e_1_3_2_26_2","first-page":"719","volume-title":"Proceedings of the USENIX Annual Technical Conference (USENIX ATC\u201918)","author":"Shi Hang","year":"2018","unstructured":"Hang Shi, Yong Cui, Xin Wang, Yuming Hu, Minglong Dai, Fanzhao Wang, and Kai Zheng. 2018. STMS: Improving MPTCP throughput under heterogeneous networks. In Proceedings of the USENIX Annual Technical Conference (USENIX ATC\u201918). 719\u2013730."},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","unstructured":"Congxi Song Biao Han Xiaolan Ji Yahui Li and Jinshu Su. 2023. AI-driven multipath transmission: Empowering UAV-based live streaming. IEEE Netw. (2023). Accepted for publication. 10.1109\/MNET.2023.3321521","DOI":"10.1109\/MNET.2023.3321521"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3336497"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/1943552.1943572"},{"key":"e_1_3_2_30_2","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton Richard S.","year":"2018","unstructured":"Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction. MIT Press."},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524273.3532889"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSAC.2020.3000365"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/TWC.2021.3081498"},{"key":"e_1_3_2_34_2","first-page":"1648","volume-title":"Proceedings of the IEEE Conference on Computer Communications (INFOCOM\u201919)","author":"Zhang Han","year":"2019","unstructured":"Han Zhang, Wenzhong Li, Shaohua Gao, Xiaoliang Wang, and Baoliu Ye. 2019. ReLeS: A neural adaptive multipath scheduler based on deep reinforcement learning. In Proceedings of the IEEE Conference on Computer Communications (INFOCOM\u201919). IEEE, 1648\u20131656."},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNET.2022.3167713"},{"key":"e_1_3_2_36_2","first-page":"418","volume-title":"Proceedings of the ACM SIGCOMM Conference","author":"Zheng Zhilong","year":"2021","unstructured":"Zhilong Zheng, Yunfei Ma, Yanmei Liu, Furong Yang, Zhenyu Li, Yuanbo Zhang, Jiuhai Zhang, Wei Shi, Wentao Chen, Ding Li et\u00a0al. 2021. Xlink: Qoe-driven multi-path QUIC transport in large-scale video services. In Proceedings of the ACM SIGCOMM Conference. 418\u2013432."}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3649139","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3649139","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:50:01Z","timestamp":1750287001000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3649139"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,16]]},"references-count":35,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2024,7,31]]}},"alternative-id":["10.1145\/3649139"],"URL":"https:\/\/doi.org\/10.1145\/3649139","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,16]]},"assertion":[{"value":"2023-08-13","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-02-08","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-05-16","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}