{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,8]],"date-time":"2026-01-08T23:52:15Z","timestamp":1767916335623,"version":"3.49.0"},"reference-count":60,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2022,7,27]],"date-time":"2022-07-27T00:00:00Z","timestamp":1658880000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,7,27]],"date-time":"2022-07-27T00:00:00Z","timestamp":1658880000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100007109","name":"Universit\u00e0 degli Studi di Ferrara","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100007109","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Netw Syst Manage"],"published-print":{"date-parts":[[2022,10]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Multi-access edge computing (MEC) is a key enabler to fulfill the promises of a new generation of immersive and low-latency services in 5G and Beyond networks. MEC represents a defining function of 5G, offering significant computational power at a reduced latency, allowing to augment the capabilities of user equipments while preserving their battery life. However, the demands generated by a plethora of innovative and concurrent IT services requiring high quality of service and quality of experience levels will likely overwhelm the\u2014albeit considerable\u2014resources available in 5G and Beyond scenarios. To take full advantage of its potential, MEC needs to be paired with innovative resource management solutions capable of effectively addressing the highly dynamic aspects of the scenario and of properly considering the heterogeneous and ever-changing nature of next generation IT services, prioritizing the assignment of resources in a highly dynamic and contextual fashion. This calls for the adoption of Artificial Intelligence based tools, implementing self-* approaches capable of learning the best resource management strategy to adapt to the ever changing conditions. In this paper, we present MECForge, a novel solution based on deep reinforcement learning that considers the maximization of total value-of-information delivered to end-user as a coherent and comprehensive resource management criterion. The experimental evaluation we conducted in a simulated but realistic environment shows how the Deep Q-Network based algorithm implemented by MECForge is capable of learning effective autonomous resource management policies that allocate service components to maximize the overall value delivered to the end-users.<\/jats:p>","DOI":"10.1007\/s10922-022-09672-6","type":"journal-article","created":{"date-parts":[[2022,8,12]],"date-time":"2022-08-12T10:03:20Z","timestamp":1660298600000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Value is King: The MECForge Deep Reinforcement Learning Solution for Resource Management in 5G and Beyond"],"prefix":"10.1007","volume":"30","author":[{"given":"Filippo","family":"Poltronieri","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Cesare","family":"Stefanelli","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Niranjan","family":"Suri","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7417-4455","authenticated-orcid":false,"given":"Mauro","family":"Tortonesi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2022,7,27]]},"reference":[{"issue":"4","key":"9672_CR1","doi-asserted-by":"publisher","first-page":"796","DOI":"10.1007\/s10922-020-09545-w","volume":"28","author":"MT Vega","year":"2020","unstructured":"Vega, M.T., Liaskos, C., Abadal, S., Papapetrou, E., Jain, A., Mouhouche, B., Kalem, G., Erg\u00fct, S., Mach, M., Sabol, T., Cabellos-Aparicio, A., Grimm, C., Turck, F.D., Famaey, J.: Immersive interconnected virtual and augmented reality: a 5G and IoT perspective. J. Netw. Syst. Manag. 28(4), 796\u2013826 (2020). https:\/\/doi.org\/10.1007\/s10922-020-09545-w","journal-title":"J. Netw. Syst. Manag."},{"key":"9672_CR2","doi-asserted-by":"publisher","first-page":"116974","DOI":"10.1109\/ACCESS.2020.3001277","volume":"8","author":"QV Pham","year":"2020","unstructured":"Pham, Q.V., Fang, F., Ha, V.N., Piran, M.J., Le, M., Le, L.B., Hwang, W.J., Ding, Z.: A survey of multi-access edge computing in 5G and beyond: fundamentals, technology integration, and state-of-the-art. IEEE Access 8, 116974\u2013117017 (2020). https:\/\/doi.org\/10.1109\/ACCESS.2020.3001277","journal-title":"IEEE Access"},{"key":"9672_CR3","doi-asserted-by":"publisher","first-page":"289","DOI":"10.1016\/j.sysarc.2019.02.009","volume":"98","author":"A Yousefpour","year":"2019","unstructured":"Yousefpour, A., Fung, C., Nguyen, T., Kadiyala, K., Jalali, F., Niakanlahiji, A., Kong, J., Jue, J.P.: All one needs to know about fog computing and related edge computing paradigms: a complete survey. J. Syst. Architect. 98, 289\u2013330 (2019). https:\/\/doi.org\/10.1016\/j.sysarc.2019.02.009","journal-title":"J. Syst. Architect."},{"issue":"3","key":"9672_CR4","doi-asserted-by":"publisher","first-page":"4005","DOI":"10.1109\/JIOT.2018.2876279","volume":"6","author":"X Chen","year":"2019","unstructured":"Chen, X., Zhang, H., Wu, C., Mao, S., Ji, Y., Bennis, M.: Optimized computation offloading performance in virtual edge computing systems via deep reinforcement learning. IEEE Internet Things J. 6(3), 4005\u20134018 (2019). https:\/\/doi.org\/10.1109\/JIOT.2018.2876279","journal-title":"IEEE Internet Things J."},{"issue":"1","key":"9672_CR5","doi-asserted-by":"publisher","DOI":"10.1002\/nem.2156","volume":"32","author":"F Poltronieri","year":"2022","unstructured":"Poltronieri, F., Tortonesi, M., Morelli, A., Stefanelli, C., Suri, N.: A value-of-information-based management framework for fog services. Int. J. Netw. Manag. 32(1), e2156 (2022). https:\/\/doi.org\/10.1002\/nem.2156","journal-title":"Int. J. Netw. Manag."},{"issue":"3","key":"9672_CR6","doi-asserted-by":"publisher","first-page":"627","DOI":"10.1007\/s10922-020-09522-3","volume":"28","author":"A Chiha","year":"2020","unstructured":"Chiha, A., der Wee, M.V., Colle, D., Verbrugge, S.: Network slicing cost allocation model. J. Netw. Syst. Manag. 28(3), 627\u2013659 (2020). https:\/\/doi.org\/10.1007\/s10922-020-09522-3","journal-title":"J. Netw. Syst. Manag."},{"key":"9672_CR7","doi-asserted-by":"publisher","DOI":"10.1007\/s10922-021-09636-2","author":"S Khan","year":"2022","unstructured":"Khan, S., Khan, S., Ali, Y., Khalid, M., Ullah, Z., Mumtaz, S.: Highly accurate and reliable wireless network slicing in 5th generation networks: a hybrid deep learning approach. J. Netw. Syst. Manag. (2022). https:\/\/doi.org\/10.1007\/s10922-021-09636-2","journal-title":"J. Netw. Syst. Manag."},{"key":"9672_CR8","unstructured":"Zhao, Y., Zhai, W., Zhao, J., Zhang, T., Sun, S., Niyato, D., Lam, K.Y.: A Comprehensive Survey of 6G Wireless Communications (2021)"},{"issue":"5","key":"9672_CR9","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1109\/MWC.001.2000032","volume":"27","author":"M Camelo","year":"2020","unstructured":"Camelo, M., Mennes, R., Shahid, A., Struye, J., Donato, C., Jabandzic, I., Giannoulis, S., Mahfoudhi, F., Maddala, P., Seskar, I., Moerman, I., Latre, S.: An AI-based incumbent protection system for collaborative intelligent radio networks. IEEE Wirel. Commun. 27(5), 16\u201323 (2020). https:\/\/doi.org\/10.1109\/MWC.001.2000032","journal-title":"IEEE Wirel. Commun."},{"issue":"4","key":"9672_CR10","doi-asserted-by":"publisher","first-page":"751","DOI":"10.1007\/s10922-020-09525-0","volume":"28","author":"MF Zhani","year":"2020","unstructured":"Zhani, M.F., ElBakoury, H.: FlexNGIA: a flexible internet architecture for the next-generation tactile internet. J. Netw. Syst. Manag. 28(4), 751\u2013795 (2020). https:\/\/doi.org\/10.1007\/s10922-020-09525-0","journal-title":"J. Netw. Syst. Manag."},{"issue":"4","key":"9672_CR11","doi-asserted-by":"publisher","first-page":"286","DOI":"10.1109\/MNET.011.2000607","volume":"35","author":"M Bagaa","year":"2021","unstructured":"Bagaa, M., Taleb, T., Riekki, J., Song, J.: Collaborative cross system AI: toward 5G system and beyond. IEEE Netw. 35(4), 286\u2013294 (2021). https:\/\/doi.org\/10.1109\/MNET.011.2000607","journal-title":"IEEE Netw."},{"key":"9672_CR12","doi-asserted-by":"publisher","DOI":"10.1007\/s10922-021-09629-1","author":"M Moradi","year":"2022","unstructured":"Moradi, M., Ahmadi, M., Nikbazm, R.: Comparison of machine learning techniques for VNF resource requirements prediction in NFV. J. Netw. Syst. Manag. (2022). https:\/\/doi.org\/10.1007\/s10922-021-09629-1","journal-title":"J. Netw. Syst. Manag."},{"key":"9672_CR13","doi-asserted-by":"publisher","DOI":"10.1007\/s10922-021-09625-5","author":"M Abusubaih","year":"2022","unstructured":"Abusubaih, M.: Intelligent wireless networks: challenges and future research topics. J. Netw. Syst. Manag. (2022). https:\/\/doi.org\/10.1007\/s10922-021-09625-5","journal-title":"J. Netw. Syst. Manag."},{"key":"9672_CR14","doi-asserted-by":"publisher","DOI":"10.1007\/s10922-020-09581-6","author":"T Goethals","year":"2021","unstructured":"Goethals, T., Turck, F.D., Volckaert, B.: Self-organizing fog support services for responsive edge computing. J. Netw. Syst. Manag. (2021). https:\/\/doi.org\/10.1007\/s10922-020-09581-6","journal-title":"J. Netw. Syst. Manag."},{"key":"9672_CR15","doi-asserted-by":"crossref","unstructured":"Arulkumaran, K., Deisenroth, M.P., Brundage, M., Bharath, A.A.: A Brief Survey of Deep Reinforcement Learning. arXiv preprint arXiv:1708.05866 (2017)","DOI":"10.1109\/MSP.2017.2743240"},{"key":"9672_CR16","doi-asserted-by":"publisher","first-page":"95032","DOI":"10.1109\/ACCESS.2020.2995456","volume":"8","author":"R Mennes","year":"2020","unstructured":"Mennes, R., De Figueiredo, F., Latr\u00e9, S.: Multi-agent deep learning for multi-channel access in slotted wireless networks. IEEE Access 8, 95032\u201395045 (2020). https:\/\/doi.org\/10.1109\/ACCESS.2020.2995456","journal-title":"IEEE Access"},{"issue":"2","key":"9672_CR17","doi-asserted-by":"publisher","first-page":"1387","DOI":"10.1109\/JIOT.2019.2955035","volume":"7","author":"M Camelo","year":"2020","unstructured":"Camelo, M., Claeys, M., Latre, S.: Parallel reinforcement learning with minimal communication overhead for IoT environments. IEEE Internet Things J. 7(2), 1387\u20131400 (2020). https:\/\/doi.org\/10.1109\/JIOT.2019.2955035","journal-title":"IEEE Internet Things J."},{"key":"9672_CR18","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2021.103535","volume":"299","author":"D Silver","year":"2021","unstructured":"Silver, D., Singh, S., Precup, D., Sutton, R.S.: Reward is enough. Artif. Intell. 299, 103535 (2021). https:\/\/doi.org\/10.1016\/j.artint.2021.103535","journal-title":"Artif. Intell."},{"issue":"10","key":"9672_CR19","doi-asserted-by":"publisher","first-page":"39","DOI":"10.1109\/MCOM.2015.7295461","volume":"53","author":"N Suri","year":"2015","unstructured":"Suri, N., Benincasa, G., Lenzi, R., Tortonesi, M., Stefanelli, C., Sadler, L.: Exploring value-of-information-based approaches to support effective communications in tactical networks. IEEE Commun. Mag. 53(10), 39\u201345 (2015). https:\/\/doi.org\/10.1109\/MCOM.2015.7295461","journal-title":"IEEE Commun. Mag."},{"key":"9672_CR20","doi-asserted-by":"publisher","DOI":"10.1109\/TETC.2019.2891716","author":"S Bharti","year":"2019","unstructured":"Bharti, S., Pattanaik, K.K., Bellavista, P.: Value of information based sensor ranking for efficient sensor service allocation in service oriented wireless sensor networks. IEEE Trans. Emerg. Topics Comput. (2019). https:\/\/doi.org\/10.1109\/TETC.2019.2891716","journal-title":"IEEE Trans. Emerg. Topics Comput."},{"key":"9672_CR21","doi-asserted-by":"crossref","unstructured":"Poltronieri, F., Tortonesi, M., Morelli, A., Stefanelli, C., Suri, N.: Value of Information based Optimal Service Fabric Management for Fog Computing. In: NOMS 2020\u20132020 IEEE\/IFIP Network Operations and Management Symposium, pp. 1\u20139 (2020)","DOI":"10.1109\/NOMS47738.2020.9110359"},{"key":"9672_CR22","doi-asserted-by":"crossref","unstructured":"Poltronieri, F., Tortonesi, M., Stefanelli, C., Suri, N.: Reinforcement Learning for value-based Placement of Fog Services. In: IM 2021 IFIP\/IEEE International Symposium on Integrated Network Management, pp. 1\u20137. IFIP\/IEEE (2021)","DOI":"10.1109\/NOMS54207.2022.9789944"},{"key":"9672_CR23","volume-title":"Reinforcement Learning: An Introduction","author":"RS Sutton","year":"2018","unstructured":"Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. Bradford Books, West Yorkshire (2018)","edition":"2"},{"issue":"4","key":"9672_CR24","doi-asserted-by":"publisher","first-page":"1318","DOI":"10.1109\/TNSM.2019.2947905","volume":"16","author":"PTA Quang","year":"2019","unstructured":"Quang, P.T.A., Hadjadj-Aoul, Y., Outtagarts, A.: A deep reinforcement learning approach for VNF forwarding graph embedding. IEEE Trans. Netw. Serv. Manag. 16(4), 1318\u20131331 (2019). https:\/\/doi.org\/10.1109\/TNSM.2019.2947905","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"issue":"2","key":"9672_CR25","doi-asserted-by":"publisher","first-page":"864","DOI":"10.1109\/TNSM.2020.2971543","volume":"17","author":"H Yao","year":"2020","unstructured":"Yao, H., Ma, S., Wang, J., Zhang, P., Jiang, C., Guo, S.: A continuous-decision virtual network embedding scheme relying on reinforcement learning. IEEE Trans. Netw. Serv. Manag. 17(2), 864\u2013875 (2020). https:\/\/doi.org\/10.1109\/TNSM.2020.2971543","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"issue":"3","key":"9672_CR26","doi-asserted-by":"publisher","first-page":"1337","DOI":"10.1109\/TNSM.2020.3000274","volume":"17","author":"A Kaur","year":"2020","unstructured":"Kaur, A., Kumar, K.: Energy-efficient resource allocation in cognitive radio networks under cooperative multi-agent model-free reinforcement learning schemes. IEEE Trans. Netw. Service Manag. 17(3), 1337\u20131348 (2020). https:\/\/doi.org\/10.1109\/TNSM.2020.3000274","journal-title":"IEEE Trans. Netw. Service Manag."},{"key":"9672_CR27","doi-asserted-by":"publisher","first-page":"2830","DOI":"10.3390\/s18092830","volume":"18","author":"L Mai","year":"2018","unstructured":"Mai, L., Dao, N.N., Park, M.: Real-time task assignment approach leveraging reinforcement learning with evolution strategies for long-term latency minimization in fog computing. Sensors 18, 2830 (2018). https:\/\/doi.org\/10.3390\/s18092830","journal-title":"Sensors"},{"key":"9672_CR28","doi-asserted-by":"publisher","unstructured":"Wei, F., Feng, G., Sun, Y., Wang, Y., Liang, Y.C.: Dynamic Network Slice Reconfiguration by Exploiting Deep Reinforcement Learning (2020). https:\/\/doi.org\/10.1109\/ICC40277.2020.9148848.","DOI":"10.1109\/ICC40277.2020.9148848."},{"issue":"4","key":"9672_CR29","doi-asserted-by":"publisher","DOI":"10.1002\/nem.2100.E2100nem.2100","volume":"30","author":"T De Schepper","year":"2020","unstructured":"De Schepper, T., Camelo, M., Famaey, J., Latr\u00e9, S.: Traffic classification at the radio spectrum level using deep learning models trained with synthetic data. Int. J. Netw. Manag. 30(4), e2100 (2020). https:\/\/doi.org\/10.1002\/nem.2100.E2100nem.2100","journal-title":"Int. J. Netw. Manag."},{"key":"9672_CR30","doi-asserted-by":"publisher","unstructured":"Soto, P., Camelo, M., Fontaine, J., Girmay, M., Shahid, A., Maglogiannis, V., De\u00a0Poorter, E., Moerman, I., Botero, J.F., Latr\u00e9, S.: Augmented Wi-Fi: An AI-based Wi-Fi Management Framework for Wi-Fi\/LTE Coexistence. In: 2020 16th International Conference on Network and Service Management (CNSM), pp. 1\u20139 (2020). https:\/\/doi.org\/10.23919\/CNSM50824.2020.9269064","DOI":"10.23919\/CNSM50824.2020.9269064"},{"key":"9672_CR31","doi-asserted-by":"publisher","DOI":"10.3390\/s21134321","author":"P Soto","year":"2021","unstructured":"Soto, P., Camelo, M., Mets, K., Wilhelmi, F., G\u00f3ez, D., Fletscher, L.A., Gaviria, N., Hellinckx, P., Botero, J.F., Latr\u00e9, S.: ATARI: a graph convolutional neural network approach for performance prediction in next-generation WLANs. Sensors (2021). https:\/\/doi.org\/10.3390\/s21134321","journal-title":"Sensors"},{"issue":"4","key":"9672_CR32","doi-asserted-by":"publisher","first-page":"3133","DOI":"10.1109\/COMST.2019.2916583","volume":"21","author":"NC Luong","year":"2019","unstructured":"Luong, N.C., Hoang, D.T., Gong, S., Niyato, D., Wang, P., Liang, Y.C., Kim, D.I.: Applications of deep reinforcement learning in communications and networking: a survey. IEEE Commun. Surv. Tutor. 21(4), 3133\u20133174 (2019). https:\/\/doi.org\/10.1109\/COMST.2019.2916583","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"9672_CR33","doi-asserted-by":"publisher","first-page":"74429","DOI":"10.1109\/ACCESS.2018.2881964","volume":"6","author":"R Li","year":"2018","unstructured":"Li, R., Zhao, Z., Sun, Q., Chih-Lin, I., Yang, C., Chen, X., Zhao, M., Zhang, H.: Deep reinforcement learning for resource management in network slicing. IEEE Access 6, 74429\u201374441 (2018). https:\/\/doi.org\/10.1109\/ACCESS.2018.2881964","journal-title":"IEEE Access"},{"key":"9672_CR34","doi-asserted-by":"publisher","unstructured":"Koo, J., Mendiratta, V.B., Rahman, M.R., Walid, A.: Deep reinforcement learning for network slicing with heterogeneous resource requirements and time varying traffic dynamics. In: 2019 15th International Conference on Network and Service Management (CNSM), pp. 1\u20135 (2019). https:\/\/doi.org\/10.23919\/CNSM46954.2019.9012702","DOI":"10.23919\/CNSM46954.2019.9012702"},{"key":"9672_CR35","doi-asserted-by":"publisher","first-page":"340","DOI":"10.1007\/s10922-019-09509-9","volume":"28","author":"R Montero","year":"2020","unstructured":"Montero, R., Agraz, F., Pag\u00e8s, A., Spadaro, S.: Enabling multi-segment 5G service provisioning and maintenance through network slicing. J. Netw. Syst. Manag. 28, 340\u2013366 (2020)","journal-title":"J. Netw. Syst. Manag."},{"key":"9672_CR36","doi-asserted-by":"publisher","unstructured":"Liu, Q., Han, T., Zhang, N., Wang, Y.: DeepSlicing: Deep Reinforcement Learning Assisted Resource Allocation for Network Slicing. In: GLOBECOM 2020 - 2020 IEEE Global Communications Conference, pp. 1\u20136 (2020). https:\/\/doi.org\/10.1109\/GLOBECOM42002.2020.9322106","DOI":"10.1109\/GLOBECOM42002.2020.9322106"},{"key":"9672_CR37","doi-asserted-by":"publisher","unstructured":"Liu, Q., Han, T., Moges, E.: EdgeSlice: Slicing Wireless Edge Computing Network with Decentralized Deep Reinforcement Learning. In: 2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS), pp. 234\u2013244 (2020). https:\/\/doi.org\/10.1109\/ICDCS47774.2020.00028","DOI":"10.1109\/ICDCS47774.2020.00028"},{"key":"9672_CR38","unstructured":"Nakanoya, M., Sato, Y., Shimonishi, H.: Environment-Adaptive Sizing and Placement of NFV Service Chains with Accelerated Reinforcement Learning. In: 2019 IFIP\/IEEE Symposium on Integrated Network and Service Management (IM), pp. 36\u201344 (2019)"},{"key":"9672_CR39","unstructured":"Dab, B., Aitsaadi, N., Langar, R.: Q-Learning Algorithm for Joint Computation Offloading and Resource Allocation in Edge Cloud. In: 2019 IFIP\/IEEE Symposium on Integrated Network and Service Management (IM), pp. 45\u201352 (2019)"},{"issue":"1","key":"9672_CR40","doi-asserted-by":"publisher","first-page":"118","DOI":"10.1109\/TNSM.2021.3049298","volume":"18","author":"L Wang","year":"2021","unstructured":"Wang, L., Mao, W., Zhao, J., Xu, Y.: DDQP: a double deep Q-learning approach to online fault-tolerant SFC placement. IEEE Trans. Netw. Serv. Manag. 18(1), 118\u2013132 (2021). https:\/\/doi.org\/10.1109\/TNSM.2021.3049298","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"issue":"2","key":"9672_CR41","doi-asserted-by":"publisher","first-page":"304","DOI":"10.1109\/JSAC.2019.2959263","volume":"38","author":"JS Pujol Roig","year":"2020","unstructured":"Pujol Roig, J.S., Gutierrez-Estevez, D.M., G\u00fcnd\u00fcz, D.: Management and orchestration of virtual network functions via deep reinforcement learning. IEEE J. Sel. Areas Commun. 38(2), 304\u2013317 (2020). https:\/\/doi.org\/10.1109\/JSAC.2019.2959263","journal-title":"IEEE J. Sel. Areas Commun."},{"issue":"1","key":"9672_CR42","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1016\/j.dcan.2018.10.003","volume":"5","author":"L Huang","year":"2019","unstructured":"Huang, L., Feng, X., Zhang, C., Qian, L., Wu, Y.: Deep reinforcement learning-based joint task offloading and bandwidth allocation for multi-user mobile edge computing. Digit. Commun. Netw. 5(1), 10\u201317 (2019). https:\/\/doi.org\/10.1016\/j.dcan.2018.10.003","journal-title":"Digit. Commun. Netw."},{"key":"9672_CR43","doi-asserted-by":"publisher","unstructured":"Baek, J., Kaddoum, G., Garg, S., Kaur, K., Gravel, V.: Managing Fog Networks using Reinforcement Learning Based Load Balancing Algorithm. In: 2019 IEEE Wireless Communications and Networking Conference (WCNC), pp. 1\u20137 (2019). https:\/\/doi.org\/10.1109\/WCNC.2019.8885745","DOI":"10.1109\/WCNC.2019.8885745"},{"key":"9672_CR44","doi-asserted-by":"publisher","first-page":"588","DOI":"10.1016\/j.ast.2019.06.024","volume":"92","author":"X Zhao","year":"2019","unstructured":"Zhao, X., Zong, Q., Tian, B., Zhang, B., You, M.: Fast task allocation for heterogeneous unmanned aerial vehicles through reinforcement learning. Aerosp. Sci. Technol. 92, 588\u2013594 (2019). https:\/\/doi.org\/10.1016\/j.ast.2019.06.024","journal-title":"Aerosp. Sci. Technol."},{"key":"9672_CR45","doi-asserted-by":"publisher","DOI":"10.1109\/TNSM.2021.3133354","author":"G Davoli","year":"2021","unstructured":"Davoli, G., Cerroni, W., Borsatti, D., Valieri, M., Tarchi, D., Raffaelli, C.: A fog computing orchestrator architecture with service model awareness. IEEE Trans. Netw. Service Manag. (2021). https:\/\/doi.org\/10.1109\/TNSM.2021.3133354","journal-title":"IEEE Trans. Netw. Service Manag."},{"key":"9672_CR46","unstructured":"Multi-access Edge Computing (MEC); General principles, patterns and common aspects of MEC Service APIs. Tech. rep., European Telecommunications Standards Institute (ETSI) (2021)"},{"issue":"2","key":"9672_CR47","doi-asserted-by":"publisher","first-page":"1110","DOI":"10.1109\/TNSM.2019.2960849","volume":"17","author":"I Com\u015fa","year":"2020","unstructured":"Com\u015fa, I., Trestian, R., Muntean, G., Ghinea, G.: 5MART: a 5G SMART scheduling framework for optimizing QoS through reinforcement learning. IEEE Trans. Netw. Service Manag. 17(2), 1110\u20131124 (2020). https:\/\/doi.org\/10.1109\/TNSM.2019.2960849","journal-title":"IEEE Trans. Netw. Service Manag."},{"issue":"5","key":"9672_CR48","doi-asserted-by":"publisher","first-page":"e1973","DOI":"10.1002\/nem.1973.E1973nem.1973","volume":"27","author":"Z Guizani","year":"2017","unstructured":"Guizani, Z., Hamdi, N.: CRAN, H-CRAN, and F-RAN for 5G systems: key capabilities and recent advances. Int. J. Netw. Manag. 27(5), e1973 (2017). https:\/\/doi.org\/10.1002\/nem.1973.E1973nem.1973","journal-title":"Int. J. Netw. Manag."},{"issue":"1","key":"9672_CR49","doi-asserted-by":"publisher","first-page":"570","DOI":"10.1109\/TNSM.2021.3052808","volume":"18","author":"I Labriji","year":"2021","unstructured":"Labriji, I., Meneghello, F., Cecchinato, D., Sesia, S., Perraud, E., Strinati, E., Rossi, M.: Mobility aware and dynamic migration of MEC services for the internet of vehicles. IEEE Trans. Netw. Service Manag. 18(1), 570\u2013584 (2021). https:\/\/doi.org\/10.1109\/TNSM.2021.3052808","journal-title":"IEEE Trans. Netw. Service Manag."},{"issue":"7540","key":"9672_CR50","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","volume":"518","author":"V Mnih","year":"2015","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., Hassabis, D.: Human-level control through deep reinforcement learning. Nature 518(7540), 529\u2013533 (2015). https:\/\/doi.org\/10.1038\/nature14236","journal-title":"Nature"},{"issue":"1","key":"9672_CR51","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1109\/TSSC.1966.300074","volume":"2","author":"RA Howard","year":"1966","unstructured":"Howard, R.A.: Information value theory. IEEE Trans. Syst. Sci. Cybernet. 2(1), 22\u201326 (1966). https:\/\/doi.org\/10.1109\/TSSC.1966.300074","journal-title":"IEEE Trans. Syst. Sci. Cybernet."},{"issue":"9","key":"9672_CR52","doi-asserted-by":"publisher","first-page":"6419","DOI":"10.1109\/TIT.2018.2818739","volume":"64","author":"C Kam","year":"2018","unstructured":"Kam, C., Kompella, S., Nguyen, G.D., Wieselthier, J.E., Ephremides, A.: On the age of information with packet deadlines. IEEE Trans. Inf. Theory 64(9), 6419\u20136428 (2018). https:\/\/doi.org\/10.1109\/TIT.2018.2818739","journal-title":"IEEE Trans. Inf. Theory"},{"key":"9672_CR53","doi-asserted-by":"publisher","first-page":"66707","DOI":"10.1109\/ACCESS.2020.2984930","volume":"8","author":"P Bellavista","year":"2020","unstructured":"Bellavista, P., Giannelli, C., Montenero, D.D.P., Poltronieri, F., Stefanelli, C., Tortonesi, M.: HOlistic pRocessing and NETworking (HORNET): an integrated solution for IoT-based fog computing services. IEEE Access 8, 66707\u201366721 (2020). https:\/\/doi.org\/10.1109\/ACCESS.2020.2984930","journal-title":"IEEE Access"},{"key":"9672_CR54","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-15-4095-0","volume-title":"Deep Reinforcement Learning","author":"H Dong","year":"2020","unstructured":"Dong, H., Dong, H., Ding, Z., Zhang, S., Chang: Deep Reinforcement Learning. Springer, Singapore (2020)"},{"key":"9672_CR55","volume-title":"Markov decision processes: discrete stochastic dynamic programming","author":"ML Puterman","year":"2014","unstructured":"Puterman, M.L.: Markov decision processes: discrete stochastic dynamic programming. Wiley, Hoboken (2014)"},{"key":"9672_CR56","doi-asserted-by":"publisher","first-page":"86769","DOI":"10.1109\/ACCESS.2019.2923610","volume":"7","author":"Q Qi","year":"2019","unstructured":"Qi, Q., Tao, F.: A smart manufacturing service system based on edge computing, fog computing, and cloud computing. IEEE Access 7, 86769\u201386777 (2019). https:\/\/doi.org\/10.1109\/ACCESS.2019.2923610","journal-title":"IEEE Access"},{"key":"9672_CR57","doi-asserted-by":"publisher","DOI":"10.1109\/TCSS.2021.3068369","author":"K Zhang","year":"2021","unstructured":"Zhang, K., Cao, J., Maharjan, S., Zhang, Y.: Digital twin empowered content caching in social-aware vehicular edge networks. IEEE Trans. Comput. Social Syst. (2021). https:\/\/doi.org\/10.1109\/TCSS.2021.3068369","journal-title":"IEEE Trans. Comput. Social Syst."},{"issue":"3","key":"9672_CR58","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1109\/MCOM.001.1900637","volume":"58","author":"F Wilhelmi","year":"2020","unstructured":"Wilhelmi, F., Barrachina-Munoz, S., Bellalta, B., Cano, C., Jonsson, A., Ram, V.: A flexible machine-learning-aware architecture for future WLANs. IEEE Commun. Mag. 58(3), 25\u201331 (2020). https:\/\/doi.org\/10.1109\/MCOM.001.1900637","journal-title":"IEEE Commun. Mag."},{"key":"9672_CR59","doi-asserted-by":"publisher","unstructured":"Poltronieri, F., Stefanelli, C., Suri, N., Tortonesi, M.: Phileas: A Simulation-based Approach for the Evaluation of Value-based Fog Services. In: 2018 IEEE 23rd International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD), pp. 1\u20136 (2018). https:\/\/doi.org\/10.1109\/CAMAD.2018.8514969","DOI":"10.1109\/CAMAD.2018.8514969"},{"issue":"3","key":"9672_CR60","doi-asserted-by":"publisher","first-page":"668","DOI":"10.1109\/JSAC.2019.2894306","volume":"37","author":"HA Alameddine","year":"2019","unstructured":"Alameddine, H.A., Sharafeddine, S., Sebbah, S., Ayoubi, S., Assi, C.: Dynamic task offloading and scheduling for low-latency IoT services in multi-access edge computing. IEEE J. Sel. Areas Commun. 37(3), 668\u2013682 (2019). https:\/\/doi.org\/10.1109\/JSAC.2019.2894306","journal-title":"IEEE J. Sel. Areas Commun."}],"container-title":["Journal of Network and Systems Management"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10922-022-09672-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10922-022-09672-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10922-022-09672-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,1]],"date-time":"2024-10-01T08:52:09Z","timestamp":1727772729000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10922-022-09672-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,27]]},"references-count":60,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,10]]}},"alternative-id":["9672"],"URL":"https:\/\/doi.org\/10.1007\/s10922-022-09672-6","relation":{},"ISSN":["1064-7570","1573-7705"],"issn-type":[{"value":"1064-7570","type":"print"},{"value":"1573-7705","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,27]]},"assertion":[{"value":"2 July 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 May 2022","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 July 2022","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 July 2022","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"63"}}