{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,7,25]],"date-time":"2024-07-25T07:11:00Z","timestamp":1721891460543},"reference-count":27,"publisher":"National Library of Serbia","issue":"1","license":[{"start":{"date-parts":[[2023,1,1]],"date-time":"2023-01-01T00:00:00Z","timestamp":1672531200000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["ComSIS","COMPUT SCI INF SYST","COMPUT SCI INFORM SY","COMPUTER SCI INFORM","COMSIS J"],"published-print":{"date-parts":[[2023]]},"abstract":"<jats:p>Dynamic adaptive streaming over HTTP (DASH) has been widely used in video streaming recently. In DASH, the client downloads video chunks in order from a server. The rate adaptation function at the video client enhances the user?s quality-of-experience (QoE) by choosing a suitable quality level for each video chunk to download based on the network condition. Today networks such as content delivery networks, edge caching networks, contentcentric networks, etc. usually replicate video contents on multiple cache nodes. We study video streaming from multiple sources in this work. In multi-source streaming, video chunks may arrive out of order due to different conditions of the network paths. Hence, to guarantee a high QoE, the video client needs not only rate adaptation, but also chunk scheduling. Reinforcement learning (RL) has emerged as the state-of-the-art control method in various fields in recent years. This paper proposes two algorithms for streaming from multiple sources: RL-based adaptation with greedy scheduling (RLAGS) and RL-based adaptation and scheduling (RLAS). We also build a simulation environment for training and evaluation. The efficiency of the proposed algorithms is proved via extensive simulations with real-trace data.<\/jats:p>","DOI":"10.2298\/csis220927055n","type":"journal-article","created":{"date-parts":[[2022,12,16]],"date-time":"2022-12-16T16:49:14Z","timestamp":1671209354000},"page":"157-173","source":"Crossref","is-referenced-by-count":0,"title":["Reinforcement learning - based adaptation and scheduling methods for multi-source DASH"],"prefix":"10.2298","volume":"20","author":[{"suffix":"T.","given":"Nghia","family":"Nguyen","sequence":"first","affiliation":[{"name":"School of Computer and Engineering, International University, Ho Chi Minh City, Vietnam + Vietnam National University, Ho Chi Minh City, Vietnam"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Long","family":"Luu","sequence":"additional","affiliation":[{"name":"School of Computer and Engineering, International University, Ho Chi Minh City, Vietnam + Vietnam National University, Ho Chi Minh City, Vietnam"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"suffix":"L.","given":"Phuong","family":"Vo","sequence":"additional","affiliation":[{"name":"School of Computer and Engineering, International University, Ho Chi Minh City, Vietnam + Vietnam National University, Ho Chi Minh City, Vietnam"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"suffix":"Thanh","given":"Sang","family":"Nguyen","sequence":"additional","affiliation":[{"name":"School of Computer and Engineering, International University, Ho Chi Minh City, Vietnam + Vietnam National University, Ho Chi Minh City, Vietnam"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"suffix":"T.","given":"Cuong","family":"Do","sequence":"additional","affiliation":[{"name":"Department of Computer Engineering, Kyung Hee University, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ngoc-Thanh","family":"Nguyen","sequence":"additional","affiliation":[{"name":"Wroclaw University of Science and Technology, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1078","reference":[{"key":"ref1","unstructured":"Cisco: Cisco Visual Networking Index: Forecast and Methodology, 2016-2021."},{"key":"ref2","doi-asserted-by":"crossref","unstructured":"T. Stockhammer: Dynamic adaptive streaming over HTTP: standards and design principles. In Proceedings of the second annual ACM conference on Multimedia systems, 133-144. (2011)","DOI":"10.1145\/1943552.1943572"},{"key":"ref3","doi-asserted-by":"crossref","unstructured":"I. Sodagar: The MPEG-DASH Standard for Multimedia Streaming Over the Internet. IEEE MultiMedia, Vol. 18, Issue 4, 62-67. (2011)","DOI":"10.1109\/MMUL.2011.71"},{"key":"ref4","doi-asserted-by":"crossref","unstructured":"S. Lederer, C. M\u00a8uller and C. Timmerer: Dynamic Adaptive Streaming over HTTP Dataset. In Proceedings of the ACM Multimedia Systems Conference, 22-24. (2012) Online: https:\/\/dash.itec.aau.at\/dash-dataset\/.","DOI":"10.1145\/2155555.2155570"},{"key":"ref5","unstructured":"ISO\/IEC 23009-1:2014: Dynamic Adaptive Streaming over HTTP (DASH)- part 1: Media Description and Segments format."},{"key":"ref6","unstructured":"DASH Reference Client. Accessed: Jun. 28, 2019. [Online]. Available: https:\/\/reference.dashif.org\/dash.js\/"},{"key":"ref7","doi-asserted-by":"crossref","unstructured":"J. Jiang, V. Sekar, and H. Zhang: Improving Fairness, Efficiency, and Stability in HTTPbased Adaptive Video Streaming with FESTIVE. In Proceedings of CoNEXT. (2012)","DOI":"10.1145\/2413176.2413189"},{"key":"ref8","doi-asserted-by":"crossref","unstructured":"K. Spiteri, R. Urgaonkar, and R. K. Sitaraman: BOLA: Near-optimal bitrate adaptation for online videos. In Proceedings of 35th Annual IEEE International Conference on Computer Communications (INFOCOM). (2016)","DOI":"10.1109\/INFOCOM.2016.7524428"},{"key":"ref9","doi-asserted-by":"crossref","unstructured":"T. Y. Huang, R. Johari, N. McKeown, M. Trunnell, and M. Watson: A buffer-based approach to rate adaptation: Evidence from a large video streaming service. In Proceedings of the 2014 ACM conference on SIGCOMM, 187-198. (2014)","DOI":"10.1145\/2619239.2626296"},{"key":"ref10","doi-asserted-by":"crossref","unstructured":"Z. Li, X. Zhu, J. Gahm, R. Pan, H. Hu, A. C. Begen, and D. Oran: Probe and adapt: Rate adaptation for HTTP video streaming at scale. IEEE Journal on Selected Areas in Communications, Vol. 32, No. 4, 719-733. (2014)","DOI":"10.1109\/JSAC.2014.140405"},{"key":"ref11","doi-asserted-by":"crossref","unstructured":"Y.C. Chen, D. Towsley, and R. Khalili: MSPlayer: Multisource and multi-path video streaming. IEEE Journal on Selected Areas in Communications, Vol.34, Issue 8, 2198- 2206. (2016)","DOI":"10.1109\/JSAC.2016.2577322"},{"key":"ref12","doi-asserted-by":"crossref","unstructured":"A. Nikravesh, Y. Guo, X. Zhu, F. Qian, and Z. M. Mao: MP-H2: a Client-only Multipath Solution for HTTP\/2. In Proceedings of The 25th Annual International Conference on Mobile Computing and Networking, 1-16. (2019)","DOI":"10.1145\/3300061.3300131"},{"key":"ref13","doi-asserted-by":"crossref","unstructured":"A. Bentaleb, P.K. Yadav,W.T. Ooi, and R. Zimmermann: DQ-DASH: A Queuing Theory Approach to Distributed Adaptive Video Streaming. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Vol. 16, No. 1, 1-24. (2020)","DOI":"10.1145\/3371040"},{"key":"ref14","doi-asserted-by":"crossref","unstructured":"M. Claeys, S. Latre, J. Famaey, and F. De Turck, \u201cDesign and evaluation of a self-learning HTTP adaptive video streaming client,\u201d IEEE communications letters, vol. 18, issue 4, pp. 716-719, 2014.","DOI":"10.1109\/LCOMM.2014.020414.132649"},{"key":"ref15","doi-asserted-by":"crossref","unstructured":"H. Mao, R. Netravali, and M. Alizadeh: Neural adaptive video streaming with pensieve. In Proceedings of the Conference of the ACM Special Interest Group on Data Communication, 197-210. (2017)","DOI":"10.1145\/3098822.3098843"},{"key":"ref16","doi-asserted-by":"crossref","unstructured":"M. Gadaleta, F. Chiariotti, M. Rossi, and A. Zanella: D-DASH: A deep Q-learning framework for DASH video streaming. IEEE Transactions on Cognitive Communications and Networking, Vol. 3, Issue 4, 703-718. (2017)","DOI":"10.1109\/TCCN.2017.2755007"},{"key":"ref17","unstructured":"D. Wischik, C. Raiciu, A. Greenhalgh, and M. Handley: Design, implementation and evaluation of congestion control for multipath TCP. In Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation, Vol. 11, 8-8. (2011)"},{"key":"ref18","doi-asserted-by":"crossref","unstructured":"C. Raiciu, M. Handley, and D. Wischik: Coupled congestion control for multipath transport protocols, RFC6356. (2011)","DOI":"10.17487\/rfc6356"},{"key":"ref19","doi-asserted-by":"crossref","unstructured":"Phuong Luu Vo, Tuan Anh Le, S. Lee, C. S. Hong, B. Kim, H. Song: mReno: a practical multipath congestion control for communication networks. Computing, Vol. 96, No. 3, 189-205. (2014)","DOI":"10.1007\/s00607-013-0341-1"},{"key":"ref20","doi-asserted-by":"crossref","unstructured":"Nghia T. Nguyen, Phuong L. Vo, Thi Thanh Sang Nguyen, Quan M. Le, Cuong T. Do, and Ngoc-Thanh Nguyen: A Reinforcement Learning Framework for Multi-source Adaptive Streaming. In Proceedings of International Conference on Computational Collective Intelligence, 416-426. (2021)","DOI":"10.1007\/978-3-030-88081-1_31"},{"key":"ref21","doi-asserted-by":"crossref","unstructured":"S. Huang and S. Ontanon: A Closer Look at Invalid Action Masking in Policy Gradient Algorithms. In Proceedings of the Thirty-Fifth International Florida Artificial Intelligence Research Society Conference, (FLAIRS 2022), Florida, USA, May 15-18. (2022)","DOI":"10.32473\/flairs.v35i.130584"},{"key":"ref22","unstructured":"US Federal Communications Commission (FCC). [Online]. Available: https:\/\/data.fcc.gov\/download\/measuring-broadband-america\/2019\/data-raw-2019-sept.tar.gz"},{"key":"ref23","unstructured":"Tenth Measuring Broadband America Fixed Broadband Report [Online]. Available: Measuring Fixed Broadband - Tenth Report - Federal Communications Commission (fcc.gov)"},{"key":"ref24","doi-asserted-by":"crossref","unstructured":"D. Raca, J.J. Quinlan, A.H. Zahran, C.J. Sreenan: Beyond Throughput: a 4G LTE Dataset with Channel and Context Metrics. In Proceedings of ACM Multimedia Systems Conference (MMSys 2018), Amsterdam, The Netherlands. (2018)","DOI":"10.1145\/3204949.3208123"},{"key":"ref25","unstructured":"J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov: Proximal Policy Optimization Algorithms. arXiv preprint arXiv:1707.06347. (2017)"},{"key":"ref26","unstructured":"A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus and N. Dormann: Stable- Baselines3: Reliable Reinforcement Learning Implementations. Journal of Machine Learning Research, Vol. 22, No. 268, 1-8. (2021)"},{"key":"ref27","unstructured":"T. M. Moerland, J. Broekens, and C. M. Jonker: Model-based reinforcement learning: A survey. arXiv preprint arXiv:2006.16712. (2020)"}],"container-title":["Computer Science and Information Systems"],"original-title":[],"language":"en","deposited":{"date-parts":[[2024,7,25]],"date-time":"2024-07-25T06:30:27Z","timestamp":1721889027000},"score":1,"resource":{"primary":{"URL":"https:\/\/doiserbia.nb.rs\/Article.aspx?ID=1820-02142200055N"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023]]},"references-count":27,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023]]}},"URL":"https:\/\/doi.org\/10.2298\/csis220927055n","relation":{},"ISSN":["1820-0214","2406-1018"],"issn-type":[{"value":"1820-0214","type":"print"},{"value":"2406-1018","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023]]}}}