{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,19]],"date-time":"2026-02-19T02:21:21Z","timestamp":1771467681907,"version":"3.50.1"},"reference-count":34,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2022,10,12]],"date-time":"2022-10-12T00:00:00Z","timestamp":1665532800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Beijing Nova Program of Science and Technology","award":["Z191100001119094"],"award-info":[{"award-number":["Z191100001119094"]}]},{"name":"Beijing Nova Program of Science and Technology","award":["L202016"],"award-info":[{"award-number":["L202016"]}]},{"name":"Beijing Nova Program of Science and Technology","award":["2020C05005"],"award-info":[{"award-number":["2020C05005"]}]},{"name":"Beijing Natural Science Foundation","award":["Z191100001119094"],"award-info":[{"award-number":["Z191100001119094"]}]},{"name":"Beijing Natural Science Foundation","award":["L202016"],"award-info":[{"award-number":["L202016"]}]},{"name":"Beijing Natural Science Foundation","award":["2020C05005"],"award-info":[{"award-number":["2020C05005"]}]},{"name":"Zhejiang \u2018JIANBING\u2019 R&amp;D Project","award":["Z191100001119094"],"award-info":[{"award-number":["Z191100001119094"]}]},{"name":"Zhejiang \u2018JIANBING\u2019 R&amp;D Project","award":["L202016"],"award-info":[{"award-number":["L202016"]}]},{"name":"Zhejiang \u2018JIANBING\u2019 R&amp;D Project","award":["2020C05005"],"award-info":[{"award-number":["2020C05005"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Complex dynamic services and heterogeneous network environments make the asymmetrical control a curial issue to handle on the Internet. With the advent of the Internet of Things (IoT) and the fifth generation (5G), the emerging network applications lead to the explosive growth of mobile traffic while bringing forward more challenging service requirements to future radio access networks. Therefore, how to effectively allocate limited heterogeneous network resources to improve content delivery for massive application services to ensure network quality of service (QoS) becomes particularly urgent in heterogeneous network environments. To cope with the explosive mobile traffic caused by emerging Internet services, this paper designs an intelligent optimization strategy based on deep reinforcement learning (DRL) for resource allocation in heterogeneous cloud-edge-end collaboration environments. Meanwhile, the asymmetrical control problem caused by complex dynamic services and heterogeneous network environments is discussed and overcome by distributed cooperation among cloud-edge-end nodes in the system. Specifically, the multi-layer heterogeneous resource allocation problem is formulated as a maximal traffic offloading model, where content caching and request aggregation mechanisms are utilized. A novel DRL policy is proposed to improve content distribution by making cache replacement and task scheduling for arriving content requests in accordance with the information about users\u2019 history requests, in-network cache capacity, available link bandwidth and topology structure. The performance of our proposed solution and its similar counterparts are analyzed in different network conditions.<\/jats:p>","DOI":"10.3390\/sym14102120","type":"journal-article","created":{"date-parts":[[2022,10,12]],"date-time":"2022-10-12T22:45:29Z","timestamp":1665614729000},"page":"2120","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["A DRL-Driven Intelligent Optimization Strategy for Resource Allocation in Cloud-Edge-End Cooperation Environments"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3989-3964","authenticated-orcid":false,"given":"Chao","family":"Fang","sequence":"first","affiliation":[{"name":"Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China"},{"name":"Purple Mountain Laboratories, Nanjing 211111, China"}]},{"given":"Tianyi","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, Beihang University, Beijing 100191, China"}]},{"given":"Jingjing","family":"Huang","sequence":"additional","affiliation":[{"name":"Beijing Saixi Technology Development Company with Limited Liability, Beijing 100007, China"}]},{"given":"Hang","family":"Xu","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China"}]},{"given":"Zhaoming","family":"Hu","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China"}]},{"given":"Yihui","family":"Yang","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2880-3329","authenticated-orcid":false,"given":"Zhuwei","family":"Wang","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China"}]},{"given":"Zequan","family":"Zhou","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, Beihang University, Beijing 100191, China"}]},{"given":"Xiling","family":"Luo","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, Beihang University, Beijing 100191, China"}]}],"member":"1968","published-online":{"date-parts":[[2022,10,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2131","DOI":"10.1109\/COMST.2021.3106401","article-title":"Resource scheduling in edge computing: A survey","volume":"23","author":"Luo","year":"2021","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2353","DOI":"10.1109\/COMST.2018.2809670","article-title":"A survey of mobile information-centric networking: Research issues and challenges","volume":"20","author":"Fang","year":"2018","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.jnca.2019.06.006","article-title":"A comprehensive survey for scheduling techniques in cloud computing","volume":"143","author":"Kumar","year":"2019","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1145\/2831347.2831354","article-title":"Edge-Centric Computing: Vision and Challenges","volume":"45","author":"Lopez","year":"2015","journal-title":"ACM SIGCOMM Comput. Commun. Rev."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1109\/MCOM.2019.1800769","article-title":"Data-driven intelligent future network: Architecture, use cases, and challenges","volume":"57","author":"Fang","year":"2019","journal-title":"IEEE Commun. Mag."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1109\/JIOT.2017.2750180","article-title":"Mobile edge computing: A survey","volume":"5","author":"Abbas","year":"2017","journal-title":"IEEE Internet Things J."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"416","DOI":"10.1109\/COMST.2017.2771153","article-title":"A comprehensive survey on fog computing: State-of-the-art and research challenges","volume":"20","author":"Mouradian","year":"2017","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1145\/2677046.2677052","article-title":"Finding your way in the fog: Towards a comprehensive definition of fog computing","volume":"44","author":"Vaquero","year":"2014","journal-title":"ACM SIGCOMM Comput. Commun. Rev."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"6926","DOI":"10.1109\/TWC.2017.2734646","article-title":"Collaborative multi-tier caching in heterogeneous networks: Modeling, analysis, and design","volume":"16","author":"Li","year":"2017","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Li, X., Wang, X., Xiao, S., and Leung, V.C. (2015, January 8\u201312). Delay performance analysis of cooperative cell caching in future mobile networks. Proceedings of the 2015 IEEE International Conference on Communications (ICC), London, UK.","DOI":"10.1109\/ICC.2015.7249223"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2960","DOI":"10.1109\/TITS.2016.2561293","article-title":"On the serviceability of mobile vehicular cloudlets in a large-scale urban environment","volume":"17","author":"Wang","year":"2016","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"8769","DOI":"10.1109\/TVT.2018.2846232","article-title":"Efficient resource allocation for on-demand mobile-edge cloud computing","volume":"67","author":"Chen","year":"2018","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"4514","DOI":"10.1109\/TVT.2018.2790421","article-title":"Collaborative computation offloading for multiaccess edge computing over fiber\u2013wireless networks","volume":"67","author":"Guo","year":"2018","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1109\/MWC.2018.1700216","article-title":"Data-driven computing and caching in 5g networks: Architecture and delay analysis","volume":"25","author":"Chen","year":"2018","journal-title":"IEEE Wirel. Commun."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1109\/MWC.2016.1500184WC","article-title":"A data-driven architecture for personalized qoe management in 5g wireless networks","volume":"24","author":"Wang","year":"2016","journal-title":"IEEE Wirel. Commun."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1109\/MNET.101.2100054","article-title":"Intelligence-empowered mobile edge computing: Framework, issues, implementation, and outlook","volume":"35","author":"Jiang","year":"2021","journal-title":"IEEE Netw."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Li, J., Gao, H., Lv, T., and Lu, Y. (2018, January 15\u201318). Deep reinforcement learning based computation offloading and resource allocation for MEC. Proceedings of the 2018 IEEE Wireless Communications and Networking Conference (WCNC), Barcelona, Spain.","DOI":"10.1109\/WCNC.2018.8377343"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1126","DOI":"10.1109\/TCCN.2021.3116251","article-title":"On joint offloading and resource allocation: A double deep q-network approach","volume":"7","author":"Khoramnejad","year":"2021","journal-title":"IEEE Trans. Cogn. Commun. Netw."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1109\/MNET.011.2000663","article-title":"Mecc: A mobile edge collaborative caching framework empowered by deep reinforcement learning","volume":"35","author":"Xu","year":"2021","journal-title":"IEEE Netw."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Wu, G., Zhao, Y., Shen, Y., Zhang, H., Shen, S., and Yu, S. (2022, January 2\u20135). Drl-based resource allocation optimization for computation offloading in mobile edge computing. Proceedings of the IEEE INFOCOM 2022\u2014IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), New York, NY, USA.","DOI":"10.1109\/INFOCOMWKSHPS54753.2022.9798323"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Breslau, L., Cao, P., Fan, L., Phillips, G., and Shenker, S. (1999, January 21\u201325). Web caching and zipf-like distributions: Evidence and implications. Proceedings of the IEEE INFOCOM\u201999. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies, the Future Is Now (Cat. No. 99CH36320), New York, NY, USA.","DOI":"10.1109\/INFCOM.1999.749260"},{"key":"ref_22","unstructured":"Dai, J., Hu, Z., Li, B., Liu, J., and Li, B. (2012, January 25\u201330). Collaborative hierarchical caching with dynamic request routing for massive content distribution. Proceedings of the 2012 Proceedings IEEE INFOCOM, Orlando, FL, USA."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"7944","DOI":"10.1109\/TVT.2019.2917890","article-title":"Computation offloading and resource allocation for cloud assisted mobile edge computing in vehicular networks","volume":"68","author":"Zhao","year":"2019","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"16874","DOI":"10.1109\/JIOT.2022.3146239","article-title":"Deep-reinforcement-learning-based resource allocation for content distribution in fog radio access networks","volume":"9","author":"Fang","year":"2022","journal-title":"IEEE Internet Things J."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Sun, C., Li, H., Li, X., Wen, J., Xiong, Q., Wang, X., and Leung, V.C. (2020, January 25\u201328). Task offloading for end-edge-cloud orchestrated computing in mobile networks. Proceedings of the 2020 IEEE Wireless Communications and Networking Conference (WCNC), Seoul, Korea.","DOI":"10.1109\/WCNC45663.2020.9120496"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"2759","DOI":"10.1109\/TPDS.2019.2926979","article-title":"Multi-hop cooperative computation offloading for industrial iot\u2013edge\u2013cloud computing environments","volume":"30","author":"Hong","year":"2019","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"2349","DOI":"10.1109\/COMST.2020.3020854","article-title":"Architectural design alternatives based on cloud\/edge\/fog computing for connected vehicles","volume":"22","author":"Wang","year":"2020","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"19450","DOI":"10.1109\/JIOT.2022.3168036","article-title":"Towards mobility-aware computation offloading and resource allocation in end-edge-cloud orchestrated computing","volume":"9","author":"Dai","year":"2022","journal-title":"IEEE Internet Things J."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"181","DOI":"10.23919\/ICN.2020.0014","article-title":"Deep reinforcement learning for dynamic computation offloading and resource allocation in cache-assisted mobile edge computing systems","volume":"1","author":"Nath","year":"2020","journal-title":"Intell. Converg. Netw."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"3133","DOI":"10.1109\/COMST.2019.2916583","article-title":"Applications of deep reinforcement learning in communications and networking: A survey","volume":"21","author":"Luong","year":"2019","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1658","DOI":"10.1109\/TVT.2021.3055511","article-title":"Multi-agent reinforcement learning based distributed transmission in collaborative cloud-edge systems","volume":"70","author":"Xu","year":"2021","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1109\/MCOM.2014.6736753","article-title":"Cache in the air: Exploiting content caching and delivery techniques for 5g systems","volume":"52","author":"Wang","year":"2014","journal-title":"IEEE Commun. Mag."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1352","DOI":"10.1109\/TC.2001.970573","article-title":"Lrfu: A spectrum of policies that subsumes the least recently used and least frequently used policies","volume":"50","author":"Lee","year":"2001","journal-title":"IEEE Trans. Comput."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"797","DOI":"10.1109\/LCOMM.2013.022213.122741","article-title":"Energy-efficient in-network caching for content-centric networking","volume":"17","author":"Li","year":"2013","journal-title":"IEEE Commun. Lett."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/14\/10\/2120\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:50:28Z","timestamp":1760143828000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/14\/10\/2120"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,12]]},"references-count":34,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2022,10]]}},"alternative-id":["sym14102120"],"URL":"https:\/\/doi.org\/10.3390\/sym14102120","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,10,12]]}}}