{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,27]],"date-time":"2025-11-27T10:53:06Z","timestamp":1764240786656,"version":"build-2065373602"},"reference-count":21,"publisher":"MDPI AG","issue":"14","license":[{"start":{"date-parts":[[2024,7,18]],"date-time":"2024-07-18T00:00:00Z","timestamp":1721260800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Guangdong Provincial Key-Area Research and Development Program","award":["2021B0101200003"],"award-info":[{"award-number":["2021B0101200003"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Edge servers frequently manage their own offline digital twin (DT) services, in addition to caching online digital twin services. However, current research often overlooks the impact of offline caching services on memory and computation resources, which can hinder the efficiency of online service task processing on edge servers. In this study, we concentrated on service caching and task offloading within a collaborative edge computing system by emphasizing the integrated quality of service (QoS) for both online and offline edge services. We considered the resource usage of both online and offline services, along with incoming online requests. To maximize the overall QoS utility, we established an optimization objective that rewards the throughput of online services while penalizing offline services that miss their soft deadlines. We formulated this as a utility maximization problem, which was proven to be NP-hard. To tackle this complexity, we reframed the optimization problem as a Markov decision process (MDP) and introduced a joint optimization algorithm for service caching and task offloading by leveraging the deep Q-network (DQN). Comprehensive experiments revealed that our algorithm enhanced the utility by at least 14.01% compared with the baseline algorithms.<\/jats:p>","DOI":"10.3390\/s24144677","type":"journal-article","created":{"date-parts":[[2024,7,18]],"date-time":"2024-07-18T16:55:36Z","timestamp":1721321736000},"page":"4677","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Integrated Quality of Service for Offline and Online Services in Edge Networks via Task Offloading and Service Caching"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-9217-9996","authenticated-orcid":false,"given":"Chuangqiang","family":"Zhan","sequence":"first","affiliation":[{"name":"School of Information Engineering, Guangdong University of Technology, Guangzhou 510006, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-6454-4992","authenticated-orcid":false,"given":"Shaojie","family":"Zheng","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Guangdong University of Technology, Guangzhou 510006, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jingyu","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Guangdong University of Technology, Guangzhou 510006, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiachao","family":"Liang","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Guangdong University of Technology, Guangzhou 510006, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-0118-7294","authenticated-orcid":false,"given":"Xiaojie","family":"Zhou","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Guangdong University of Technology, Guangzhou 510006, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,7,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Glaessgen, E., and Stargel, D. (2012, January 23\u201326). The digital twin paradigm for future NASA and US Air Force vehicles. Proceedings of the 53rd AIAA Structures, Structural Dynamics and Materials Conference, Honolulu, HI, USA.","DOI":"10.2514\/6.2012-1818"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"729","DOI":"10.1109\/JRFID.2022.3212169","article-title":"Application of Lightweight Digital Twin System in Intelligent Transportation","volume":"6","author":"Liu","year":"2022","journal-title":"IEEE J. Radio Freq. Identif."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2063","DOI":"10.1109\/TNNLS.2018.2790388","article-title":"Applications of Deep Learning and Reinforcement Learning to Biological Data","volume":"6","author":"Mahmud","year":"2018","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2451","DOI":"10.1109\/TCC.2020.3037306","article-title":"Dependency-aware computation offloading for mobile edge computing with edge-cloud cooperation","volume":"10","author":"Chen","year":"2020","journal-title":"IEEE Trans. Cloud Comput."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1047","DOI":"10.1109\/TNET.2020.2980175","article-title":"Service placement and request routing in mec networks with storage, computation, and communication constraints","volume":"28","author":"Poularakis","year":"2020","journal-title":"IEEE Trans. Netw."},{"key":"ref_6","first-page":"4933","article-title":"Secure and latency-aware digital twin assisted resource scheduling for oretical 5G edge computing-empowered distribution grids","volume":"7","author":"Zhou","year":"2021","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"7228","DOI":"10.1109\/TII.2022.3227655","article-title":"Federated Reinforcement Learning-Based Resource Allocation for D2D-Aided Digital Twin Edge Networks in 6G Industrial IoT","volume":"19","author":"Guo","year":"2023","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Yao, M., Chen, L., Zhang, J., Huang, J., and Wu, J. (2022, January 16\u201320). Loading costaware model caching and request routing for cooperative edge inference. Proceedings of the 2022 IEEE International Conference on Communications, Seoul, Republic of Korea.","DOI":"10.1109\/ICC45855.2022.9838823"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1755","DOI":"10.1109\/TCOMM.2023.3237854","article-title":"DRL-Based Offloading for Computation Delay Minimization in Wireless-Powered Multi-Access Edge Computing","volume":"71","author":"Zheng","year":"2023","journal-title":"IEEE Trans. Commun."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Chen, L., Dai, H., Yuan, X., Huang, J., Wu, Y., and Wu, J. (2024). D-SPAC: Double-Sided Preference-Aware Carpooling of Private Cars for Maximizing Passenger Utility. IEEE Trans. Intell. Transp. Syst., 1\u201318.","DOI":"10.1109\/TITS.2024.3353545"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"He, X., and Dou, W. (2020, January 19\u201323). Offloading Deadline-aware Task in Edge Computing. Proceedings of the IEEE 13th International Conference on Cloud Computing, Beijing, China.","DOI":"10.1109\/CLOUD49709.2020.00015"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"He, X., Zheng, J., He, Q., Dai, H., Liu, B., Dou, W., and Chen, G. (2021, January 5\u201310). CONFECT: Computation Offloading for Tasks with Hard\/Soft Deadlines in Edge Computing. Proceedings of the 2021 IEEE International Conference on Web Services, Chicago, IL, USA.","DOI":"10.1109\/ICWS53863.2021.00044"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"3870","DOI":"10.1109\/TNSE.2021.3115054","article-title":"Deadline-Aware Task Offloading with Partially-Observable Deep Reinforcement Learning for Multi-Access Edge Computing","volume":"9","author":"Huang","year":"2022","journal-title":"IEEE Trans. Netw. Sci. Eng."},{"key":"ref_14","first-page":"31","article-title":"Qos-aware task offloading in fog environment using multi-agent deep reinforcement learning","volume":"7","author":"Jain","year":"2023","journal-title":"J. Netw. Syst. Manag."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2534","DOI":"10.1109\/JIOT.2023.3294887","article-title":"Task Class Partitioning for Mobile Computation Offloading","volume":"2","author":"Chen","year":"2024","journal-title":"IEEE Internet Things J."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"9408","DOI":"10.1109\/TVT.2021.3099303","article-title":"Collaborative Data Caching and Computation Offloading for Multi-Service Mobile Edge Computing","volume":"70","author":"Feng","year":"2021","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"9029","DOI":"10.1109\/TWC.2023.3267829","article-title":"Joint Service Caching and Computing Resource Allocation for Edge Computing-Enabled Networks","volume":"22","author":"Kim","year":"2022","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"5288","DOI":"10.1109\/TWC.2021.3066650","article-title":"Joint Service Caching, Computation Offloading and Resource Allocation in Mobile Edge Computing Systems","volume":"20","author":"Zhang","year":"2021","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_19","first-page":"3343","article-title":"Joint Service Caching, Resource Allocation and Computation Offloading in Three-Tier Cooperative Mobile Edge Computing System","volume":"6","author":"Wang","year":"2023","journal-title":"IEEE Trans. Netw. Sci. Eng."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"12203","DOI":"10.1109\/JIOT.2023.3333826","article-title":"Deep-Reinforcement-Learning-Based Joint Caching and Resources Allocation for Cooperative MEC","volume":"7","author":"Zhang","year":"2024","journal-title":"IEEE Internet Things J."},{"key":"ref_21","first-page":"229","article-title":"Reinforcement learning: An introduction","volume":"17","author":"Sutton","year":"1999","journal-title":"Robotica"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/14\/4677\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:19:21Z","timestamp":1760109561000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/14\/4677"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,18]]},"references-count":21,"journal-issue":{"issue":"14","published-online":{"date-parts":[[2024,7]]}},"alternative-id":["s24144677"],"URL":"https:\/\/doi.org\/10.3390\/s24144677","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2024,7,18]]}}}