{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,19]],"date-time":"2026-01-19T11:19:33Z","timestamp":1768821573204,"version":"3.49.0"},"reference-count":25,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2022,8,23]],"date-time":"2022-08-23T00:00:00Z","timestamp":1661212800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea (NRF)","doi-asserted-by":"publisher","award":["NRF-2021R1F1A1063640"],"award-info":[{"award-number":["NRF-2021R1F1A1063640"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea (NRF)","doi-asserted-by":"publisher","award":["GCU-202008450002"],"award-info":[{"award-number":["GCU-202008450002"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002631","name":"Gachon University Research Fund","doi-asserted-by":"publisher","award":["NRF-2021R1F1A1063640"],"award-info":[{"award-number":["NRF-2021R1F1A1063640"]}],"id":[{"id":"10.13039\/501100002631","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002631","name":"Gachon University Research Fund","doi-asserted-by":"publisher","award":["GCU-202008450002"],"award-info":[{"award-number":["GCU-202008450002"]}],"id":[{"id":"10.13039\/501100002631","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With the recent advances in computing devices such as smartphones and laptops, most devices are equipped with multiple network interfaces such as cellular, Wi-Fi, and Ethernet. Multipath TCP (MPTCP) has been the de facto standard for utilizing multipaths, and Multipath QUIC (MPQUIC), which is an extension of the Quick UDP Internet Connections (QUIC) protocol, has become a promising replacement due to its various advantages. The multipath scheduler, which determines the path to which each packet should be transmitted, is a key function that affects the multipath transport performance. For example, the default minRTT scheduler typically achieves good throughput, while the redundant scheduler gains low latency. While the legacy schedulers may generally give a desirable performance in some environments, however, each application renders different requirements. For example, Web applications target low latency, while video streaming applications require low jitter and high video quality. In this paper, we propose a novel MPQUIC scheduler based on deep reinforcement learning using the Deep Q-Network (DQN) that enhances the quality of multimedia streaming. Our proposal first takes into account both delay and throughput as a reward for reinforcement learning to achieve a low video chunk download time. Second, we propose a chunk manager that informs the scheduler of the video chunk information, and we also tune the learning parameters to explore new random actions adequately. Finally, we implement our new scheduler on the Linux kernel and give results using the Mininet experiments. The evaluation results show that our proposal outperforms legacy schedulers by at least 20%.<\/jats:p>","DOI":"10.3390\/s22176333","type":"journal-article","created":{"date-parts":[[2022,8,24]],"date-time":"2022-08-24T02:55:34Z","timestamp":1661309734000},"page":"6333","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Reinforcement Learning Based Multipath QUIC Scheduler for Multimedia Streaming"],"prefix":"10.3390","volume":"22","author":[{"given":"Seunghwa","family":"Lee","sequence":"first","affiliation":[{"name":"School of Computing, Gachon University, 1342 Seongnam-daero, Sujeong-gu, Seongnam-si 13120, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9520-5855","authenticated-orcid":false,"given":"Joon","family":"Yoo","sequence":"additional","affiliation":[{"name":"School of Computing, Gachon University, 1342 Seongnam-daero, Sujeong-gu, Seongnam-si 13120, Korea"}]}],"member":"1968","published-online":{"date-parts":[[2022,8,23]]},"reference":[{"key":"ref_1","unstructured":"Schumann, L., Doan, T.V., Shreedhar, T., Mok, R., and Bajpai, V. (2022). Impact of Evolving Protocols and COVID-19 on Internet Traffic Shares. arXiv."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1109\/4236.968833","article-title":"SCTP: New transport protocol for TCP\/IP","volume":"5","author":"Stewart","year":"2001","journal-title":"IEEE Internet Comput."},{"key":"ref_3","unstructured":"Wischik, D., Raiciu, C., Greenhalgh, A., and Handley, M. (April, January 30). Design, implementation and evaluation of congestion control for multipath TCP. Proceedings of the 8th USENIX Symposium on Networked Systems Design and Implementation (NSDI 11), Boston, MA, USA."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"596","DOI":"10.1109\/TNET.2014.2379698","article-title":"Multipath TCP: Analysis, design, and implementation","volume":"24","author":"Peng","year":"2014","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"De Coninck, Q., and Bonaventure, O. (2017, January 12\u201315). Multipath quic: Design and evaluation. Proceedings of the 13th International Conference on Emerging Networking Experiments and Technologies, Incheon, Korea.","DOI":"10.1145\/3143361.3143370"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Langley, A., Riddoch, A., Wilk, A., Vicente, A., Krasic, C., Zhang, D., Yang, F., Kouranov, F., Swett, I., and Iyengar, J. (2017, January 21\u201325). The quic transport protocol: Design and internet-scale deployment. Proceedings of the Conference of the ACM Special Interest Group on Data Communication, Los Angeles, CA, USA.","DOI":"10.1145\/3098822.3098842"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Frommgen, A., Erbsh\u00e4u\u00dfer, T., Buchmann, A., Zimmermann, T., and Wehrle, K. (2016, January 22\u201327). ReMP TCP: Low latency multipath TCP. Proceedings of the 2016 IEEE International Conference on Communications (ICC), Kuala Lumpur, Malaysia.","DOI":"10.1109\/ICC.2016.7510787"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Ferlin, S., Alay, \u00d6., Mehani, O., and Boreli, R. (2016, January 17\u201319). BLEST: Blocking estimation-based MPTCP scheduler for heterogeneous networks. Proceedings of the 2016 IFIP Networking Conference (IFIP Networking) and Workshops, Vienna, Austria.","DOI":"10.1109\/IFIPNetworking.2016.7497206"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Stockhammer, T. (2011, January 23\u201325). Dynamic adaptive streaming over HTTP\u2013standards and design principles. Proceedings of the Second Annual ACM Conference on Multimedia Systems, San Jose, CA, USA.","DOI":"10.1145\/1943552.1943572"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"7230","DOI":"10.1109\/TWC.2021.3081498","article-title":"A low-latency mptcp scheduler for live video streaming in mobile networks","volume":"20","author":"Xing","year":"2021","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_11","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_12","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Chao, L., Wu, C., Yoshinaga, T., Bao, W., and Ji, Y. (2021). A brief review of multipath tcp for vehicular networks. Sensors, 21.","DOI":"10.3390\/s21082793"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Nguyen, K., Golam Kibria, M., Ishizu, K., Kojima, F., and Sekiya, H. (2019). An approach to reinforce multipath TCP with path-aware information. Sensors, 19.","DOI":"10.3390\/s19030476"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Hwang, J., and Yoo, J. (2020). A memory-efficient transmission scheme for multi-homed internet-of-things (IoT) devices. Sensors, 20.","DOI":"10.3390\/s20051436"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Deng, S., Netravali, R., Sivaraman, A., and Balakrishnan, H. (2014, January 5\u20137). WiFi, LTE, or both? Measuring multi-homed wireless internet performance. Proceedings of the 2014 Conference on Internet Measurement Conference, Vancouver, BC, Canada.","DOI":"10.1145\/2663716.2663727"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Lim, Y.S., Nahum, E.M., Towsley, D., and Gibbens, R.J. (2017, January 12\u201315). ECF: An MPTCP path scheduler to manage heterogeneous paths. Proceedings of the 13th International Conference on Emerging Networking Experiments and Technologies, Incheon, Korea.","DOI":"10.1145\/3143361.3143376"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2295","DOI":"10.1109\/JSAC.2020.3000365","article-title":"Peekaboo: Learning-based multipath scheduling for dynamic heterogeneous environments","volume":"38","author":"Wu","year":"2020","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12\u201317). Deep reinforcement learning with double q-learning. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zhang, H., Li, W., Gao, S., Wang, X., and Ye, B. (May, January 29). ReLeS: A neural adaptive multipath scheduler based on deep reinforcement learning. Proceedings of the IEEE INFOCOM 2019-IEEE Conference on Computer Communications, Paris, France.","DOI":"10.1109\/INFOCOM.2019.8737649"},{"key":"ref_21","unstructured":"Braden, R. (2022, July 01). RFC1122: Requirements for Internet Hosts-Communication Layers. Available online: https:\/\/datatracker.ietf.org\/doc\/rfc1122\/."},{"key":"ref_22","unstructured":"(2022, July 01). Mininet: An Instant Virtual Network on Your Laptop (or Other PC)-Mininet. Available online: http:\/\/mininet.org\/."},{"key":"ref_23","unstructured":"(2022, July 01). Caddy-The Ultimate Server with Automatic HTTPS. Available online: https:\/\/caddyserver.com."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Vu, V.A., and Walker, B. (2020, January 12\u201314). On the latency of multipath-quic in real-time applications. Proceedings of the 2020 16th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob), Thessaloniki, Greece.","DOI":"10.1109\/WiMob50308.2020.9253402"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Rosell\u00f3, M.M. (2019, January 18\u201321). Multi-path scheduling with deep reinforcement learning. Proceedings of the 2019 European Conference on Networks and Communications (EuCNC), Valencia, Spain.","DOI":"10.1109\/EuCNC.2019.8802063"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/17\/6333\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:14:06Z","timestamp":1760141646000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/17\/6333"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,23]]},"references-count":25,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["s22176333"],"URL":"https:\/\/doi.org\/10.3390\/s22176333","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,8,23]]}}}