{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T05:13:13Z","timestamp":1781673193617,"version":"3.54.5"},"reference-count":50,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2022,3,10]],"date-time":"2022-03-10T00:00:00Z","timestamp":1646870400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Dynamic Adaptive Streaming over HTTP (DASH) is a promising scheme for improving the Quality of Experience (QoE) of users in video streaming. However, the existing schemes do not perform coordination among clients and depend on fixed heuristics. In this paper, we propose an adaptive streaming scheme with reinforcement learning in edge computing environments. The proposed scheme improves the overall QoE of clients and QoE fairness among clients based on a state-of-the-art reinforcement learning algorithm. Edge computing assistance plays a role in providing client-side observations to the mobile edge, making agents utilize this information when generating a policy for multi-client adaptive streaming. We evaluated the proposed scheme through simulation-based experiments under various network conditions. The experimental results show that the proposed scheme achieves better performance than the existing schemes.<\/jats:p>","DOI":"10.3390\/s22062171","type":"journal-article","created":{"date-parts":[[2022,3,10]],"date-time":"2022-03-10T20:19:10Z","timestamp":1646943550000},"page":"2171","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Reinforcement Learning-Based Adaptive Streaming Scheme with Edge Computing Assistance"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6092-051X","authenticated-orcid":false,"given":"Minsu","family":"Kim","sequence":"first","affiliation":[{"name":"Department of Electronics and Communications Engineering, Kwangwoon University, Seoul 01897, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0283-0900","authenticated-orcid":false,"given":"Kwangsue","family":"Chung","sequence":"additional","affiliation":[{"name":"Department of Electronics and Communications Engineering, Kwangwoon University, Seoul 01897, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,3,10]]},"reference":[{"key":"ref_1","unstructured":"(2021, July 01). Cisco Public. Cisco Annual Internet Report (2018\u20132023) White Paper. Available online: https:\/\/www.cisco.com\/c\/en\/us\/solutions\/collateral\/executive-perspectives\/annual-internet-report\/white-paper-c11-741490.html."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1842","DOI":"10.1109\/COMST.2017.2685630","article-title":"A Survey of Rate Adaptation Techniques for Dynamic Adaptive Streaming over HTTP","volume":"19","author":"Kua","year":"2017","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"432","DOI":"10.1109\/TBC.2018.2822869","article-title":"A Review of Predictive Quality of Experience Management in Video Streaming Service","volume":"64","author":"Vega","year":"2018","journal-title":"IEEE Trans. Broadcast."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"526","DOI":"10.1109\/COMST.2019.2958784","article-title":"QoE Management of Multimedia Streaming Services in Future Networks: A Tutorial and Survey","volume":"22","author":"Barakabitze","year":"2020","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3165266","article-title":"Quality of Experience-Centric Management of Adaptive Video Streaming Services: Status and Challenges","volume":"14","author":"Petrangeli","year":"2018","journal-title":"ACM Trans. Multimed. Comput. Commun. Appl."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Liu, C., Bouazizi, I., and Gabboi, M. (2011, January 23\u201325). Rate Adaptation for Adaptive HTTP Streaming. Proceedings of the Second Annual ACM Conference on Multimedia Systems, San Jose, CA, USA.","DOI":"10.1145\/1943552.1943575"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Huang, T.-Y., Johari, R., McKeown, N., Trunnell, M., and Watson, M. (2014, January 17\u201322). A Buffer-Based Approach to Rate Adaptation: Evidence from A Large Video Streaming Service. Proceedings of the 2014 ACM Conference on SIGCOMM, Chicago, IL, USA.","DOI":"10.1145\/2619239.2626296"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"719","DOI":"10.1109\/JSAC.2014.140405","article-title":"Probe and Adapt: Rate Adaptation for HTTP Video Streaming at Scale","volume":"32","author":"Li","year":"2014","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Jiang, J., Sekar, V., and Zhang, H. (2012, January 10\u201313). Improving Fairness, Efficiency, and Stability in HTTP-Based Adaptive Video Streaming with FESTIVE. Proceedings of the 8th International Conference on Emerging Networking Experiments and Technologies, New York, NY, USA.","DOI":"10.1145\/2413176.2413189"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Yin, X., Jindal, A., Sekar, V., and Sinopoli, B. (2015, January 17\u201321). A Control-Theoretic Approach for Dynamic Adaptive Video Streaming over HTTP. Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication, New York, NY, USA.","DOI":"10.1145\/2785956.2787486"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"30831","DOI":"10.1109\/ACCESS.2019.2901778","article-title":"QoE Modeling for HTTP Adaptive Video Streaming\u2014A Survey and Open Challenges","volume":"7","author":"Barman","year":"2019","journal-title":"IEEE Access"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1007\/s11235-014-9938-8","article-title":"A Cost-Effective and Real-Time QoE Evaluation Method for Multimedia Streaming Services","volume":"59","author":"Li","year":"2015","journal-title":"Telecommun. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Akhshabi, S., Anantakrishnan, L., Begen, A.C., and Dovrolis, C. (2012, January 7\u20138). What Happens When HTTP Adaptive Streaming Players Compete for Bandwidth?. Proceedings of the 22nd International Workshop on Network and Operating System Support for Digital Audio and Video, Toronto, ON, Canada.","DOI":"10.1145\/2229087.2229092"},{"key":"ref_14","unstructured":"Chen, J., Mahindra, R., Khojastepour, M.A., Rangarajan, S., and Chiang, M. (October, January 30). A Scheduling Framework for Adaptive Video Delivery over Cellular Network. Proceedings of the 19th Annual International Conference on Mobile Computing and Networking, Miami, FL, USA."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"De Vleeschauwer, D., Viswanathan, H., Beck, A., Benno, S., Li, G., and Miller, R. (2013, January 14\u201319). Optimization of HTTP Adaptive Streaming over Mobile Cellular Networks. Proceedings of the IEEE INFOCOM, Turin, Italy.","DOI":"10.1109\/INFCOM.2013.6566888"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"562","DOI":"10.1109\/COMST.2018.2862938","article-title":"A Survey on Bitrate Adaptation Schemes for Streaming Media over HTTP","volume":"21","author":"Bentaleb","year":"2019","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Bae, S., Jang, D., and Park, K. (2015, January 27\u201328). Why Is HTTP Adaptive Streaming So Hard?. Proceedings of the 6th Asia-Pacific Workshop on Systems, Tokyo, Japan.","DOI":"10.1145\/2797022.2797031"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"15530","DOI":"10.1109\/ACCESS.2021.3052552","article-title":"Balancing Quality of Experience and Traffic Volume in Adaptive Bitrate Streaming","volume":"9","author":"Kimura","year":"2021","journal-title":"IEEE Access"},{"key":"ref_19","unstructured":"Akhshabi, S., Anantakrishnan, L., Dovrolis, C., and Begen, A.C. (March, January 26). Server-Based Traffic Shaping for Stabilizing Oscillating Adaptive Streaming Players. Proceedings of the 23rd ACM Workshop on Network and Operating Systems Support for Digital Audio and Video, Oslo, Norway."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Yan, Z., Xue, J., and Chen, C.W. (2014, January 14\u201318). QoE Continuum Driven HTTP Adaptive Streaming over Multi-Client Wireless Networks. Proceedings of the 2014 IEEE International Conference on Multimedia and Expo, Chengdu, China.","DOI":"10.1109\/ICME.2014.6890309"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ramarkrishnan, S., Zhu, X., Chan, F., and Kambhatia, K. (2015, January 14\u201316). SDN Based QoE Optimization for HTTP-Based Adaptive Video Streaming. Proceedings of the 2015 IEEE International Symposium on Multimedia, Miami, FL, USA.","DOI":"10.1109\/ISM.2015.53"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Li, Z., Zhao, S., Medhi, D., and Bouazizi, I. (2016, January 27\u201330). Wireless Video Traffic Bottleneck Coordination with A DASH SAND Framework. Proceedings of the 2016 Visual Communications and Image Processing, Chengdu, China.","DOI":"10.1109\/VCIP.2016.7805471"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1109\/TCSVT.2016.2539827","article-title":"Prius: Hybrid Edge Cloud and Client Adaptation for HTTP Adaptive Streaming in Cellular Networks","volume":"27","author":"Yan","year":"2017","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"878","DOI":"10.1109\/TMC.2018.2850026","article-title":"Edge Computing Assisted Adaptive Mobile Video Streaming","volume":"18","author":"Mehrabi","year":"2019","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"129082","DOI":"10.1109\/ACCESS.2019.2940292","article-title":"Edge Computing Assisted Joint Quality Adaptation for Mobile Video Streaming","volume":"7","author":"Rahman","year":"2019","journal-title":"IEEE Access"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Aguilar-Armijo, J., Timmerer, C., and Hellwagner, H. (2021, January 4\u20137). EADAS: Edge Assisted Adaptation Scheme for HTTP Adaptive Streaming. Proceedings of the IEEE 46th Conference on Local Computer Networks, Edmonton, AB, Canada.","DOI":"10.1109\/LCN52139.2021.9524883"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1109\/TBC.2017.2781146","article-title":"On Improving Video Streaming Efficiency, Fairness, Stability, and Convergence Time Through Client-Server Cooperation","volume":"64","author":"Marai","year":"2018","journal-title":"IEEE Trans. Broadcast."},{"key":"ref_28","first-page":"184","article-title":"Definition of QoE Fairness in Shared Systems","volume":"21","author":"Heegaard","year":"2018","journal-title":"IEEE Commun. Lett."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"10787","DOI":"10.1007\/s11042-017-4917-1","article-title":"Q-FDBA: Improving QoE Fairness for Video Streaming","volume":"77","author":"Jiang","year":"2018","journal-title":"Multimed. Tools Appl."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3397227","article-title":"QoE-Fair DASH Video Streaming Using Server-Side Reinforcement Learning","volume":"16","author":"Altamimi","year":"2020","journal-title":"ACM Trans. Multimed. Comput. Commun. Appl."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Mok, R.K.P., Chan, E.W.W., Luo, X., and Chang, R.K.C. (2011, January 19). Inferring the QoE of HTTP Video Streaming from User-Viewing Activities. Proceedings of the First ACM SIGCOMM Workshop on Measurements up the Stack, Toronto, ON, Canada.","DOI":"10.1145\/2018602.2018611"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Mok, R.K.P., Chen, E.W.W., and Chang, R.K.C. (2011, January 23\u201327). Measuring the Quality of Experience of HTTP Video Streaming. Proceedings of the 12th IFIP\/IEEE International Symposium on Integrated Network Management and Workshops, Dublin, Ireland.","DOI":"10.1109\/INM.2011.5990550"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Mao, H., Netravali, R., and Alizadeh, M. (2017, January 21\u201325). Neural Adaptive Video Streaming with Pensieve. Proceedings of the Conference on the ACM Special Interest Group on Data Communication, Los Angeles, CA, USA.","DOI":"10.1145\/3098822.3098843"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3219752","article-title":"ORL-SDN: Online Reinforcement Learning for SDN-Enabled HTTP Adaptive Streaming","volume":"14","author":"Bentaleb","year":"2018","journal-title":"ACM Trans. Multimed. Comput. Commun. Appl."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"2415","DOI":"10.1109\/TCSVT.2020.3021489","article-title":"A Deep Reinforcement Learning Approach to Multiple Streams\u2019 Joint Bitrate Allocation","volume":"31","author":"Zhang","year":"2021","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"703","DOI":"10.1109\/TCCN.2017.2755007","article-title":"D-DASH: A Deep Q-Learning Framework for DASH Video Streaming","volume":"3","author":"Gadaleta","year":"2017","journal-title":"IEEE Trans. Cogn. Commun."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"786","DOI":"10.1016\/j.ins.2021.05.012","article-title":"Reinforcement Learning-Based QoE-Oriented Dynamic Adaptive Streaming Framework","volume":"569","author":"Wei","year":"2021","journal-title":"Inf. Sci."},{"key":"ref_38","unstructured":"Mnih, V., Badi, A.P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., and Kavukcuoglu, K. (2016, January 19\u201324). Asynchronous Methods for Deep Reinforcement Learning. Proceedings of the 33rd International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_39","unstructured":"Sutton, R.S., McAllester, D., Singh, S., and Mansour, Y. (December, January 29). Policy Gradient Methods for Reinforcement Learning with Function Approximation. Proceedings of the 12th International Conference on Neural Information Processing Systems, Denver, CO, USA."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Dabney, W., and Barto, A.G. (2012, January 22\u201326). Adaptive Step-Size for Online Temporal Difference Learning. Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence, Toronto, ON, Canada.","DOI":"10.1609\/aaai.v26i1.8313"},{"key":"ref_41","unstructured":"Konda, V.R., and Tsitsiklis, J.N. (2000). Actor-Critic Algorithms. Advances in Neural Information Processing Systems, MIT Press. Available online: https:\/\/proceedings.neurips.cc\/paper\/1999\/file\/6449f44a102fde848669bdd9eb6b76fa-Paper.pdf."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Petrangeli, S., Claeys, M., Latre, S., Famaey, J., and Truck, F.D. (2014, January 5\u20139). A Multi-Agent Q-Learning-Based Framework for Achieving Fairness in HTTP Adaptive Streaming. Proceedings of the 2014 IEEE Network Operations and Management Symposium, Krakow, Poland.","DOI":"10.1109\/NOMS.2014.6838245"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"729","DOI":"10.1109\/TWC.2019.2935201","article-title":"Multi-Agent Reinforcement Learning-Based Resource Allocation for UAV Networks","volume":"19","author":"Cui","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_44","unstructured":"Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., and Isard, M. (2016, January 2\u20134). TensorFlow: A System for Large-Scale Machine Learning. Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation, Savannah, GA, USA."},{"key":"ref_45","unstructured":"TFLearn (2021, May 01). TFLearn: Deep Learning Library Featuring a Higher-Level API for TensorFlow. Available online: http:\/\/tflearn.org\/."},{"key":"ref_46","unstructured":"Federal Communications Commission (FCC) (2021, June 01). Raw Data-Measuring Broadband America Mobile Data, Available online: https:\/\/www.fcc.gov\/reports-research\/reports."},{"key":"ref_47","unstructured":"Riiser, H., Vigmostad, P., Griwodz, C., and Halvorsen, P. (March, January 28). Commute Path Bandwidth Traces from 3G Networks: Analysis and Applications. Proceedings of the 4th ACM Multimedia Systems Conference, Oslo, Norway."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"2177","DOI":"10.1109\/LCOMM.2016.2601087","article-title":"HTTP\/2-Based Adaptive Streaming of HEVC Video over 4G\/LTE Networks","volume":"20","author":"Hooft","year":"2016","journal-title":"IEEE Commun. Lett."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"3496","DOI":"10.1109\/TWC.2013.061413.121703","article-title":"Optimal Tradeoff Between Sum-Rate Efficiency and Jain\u2019s Fairness Index in Resource Allocation","volume":"12","author":"Sediq","year":"2013","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_50","unstructured":"DASH Industry Forum (2021, June 01). Reference Client 2.4.0. Available online: https:\/\/reference.dashif.org\/dash.js\/v2.4.0\/samples\/dash-if-reference-player\/index.html."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/6\/2171\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:34:34Z","timestamp":1760135674000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/6\/2171"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,10]]},"references-count":50,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2022,3]]}},"alternative-id":["s22062171"],"URL":"https:\/\/doi.org\/10.3390\/s22062171","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,10]]}}}