{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T01:54:59Z","timestamp":1760234099410,"version":"build-2065373602"},"reference-count":24,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2021,3,23]],"date-time":"2021-03-23T00:00:00Z","timestamp":1616457600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>With the emergence of various types of applications such as delay-sensitive applications, future communication networks are expected to be increasingly complex and dynamic. Network Function Virtualization (NFV) provides the necessary support towards efficient management of such complex networks, by virtualizing network functions and placing them on shared commodity servers. However, one of the critical issues in NFV is the resource allocation for the highly complex services; moreover, this problem is classified as an NP-Hard problem. To solve this problem, our work investigates the potential of Deep Reinforcement Learning (DRL) as a swift yet accurate approach (as compared to integer linear programming) for deploying Virtualized Network Functions (VNFs) under several Quality-of-Service (QoS) constraints such as latency, memory, CPU, and failure recovery requirements. More importantly, the failure recovery requirements are focused on the node-outage problem where outage can be either due to a disaster or unavailability of network topology information (e.g., due to proprietary and ownership issues). In DRL, we adopt a Deep Q-Learning (DQL) based algorithm where the primary network estimates the action-value function Q, as well as the predicted Q, highly causing divergence in Q-value\u2019s updates. This divergence increases for the larger-scale action and state-space causing inconsistency in learning, resulting in an inaccurate output. Thus, to overcome this divergence, our work has adopted a well-known approach, i.e., introducing Target Neural Networks and Experience Replay algorithms in DQL. The constructed model is simulated for two real network topologies\u2014Netrail Topology and BtEurope Topology\u2014with various capacities of the nodes (e.g., CPU core, VNFs per Core), links (e.g., bandwidth and latency), several VNF Forwarding Graph (VNF-FG) complexities, and different degrees of the nodal outage from 0% to 50%. We can conclude from our work that, with the increase in network density or nodal capacity or VNF-FG\u2019s complexity, the model took extremely high computation time to execute the desirable results. Moreover, with the rise in complexity of the VNF-FG, the resources decline much faster. In terms of the nodal outage, our model provided almost 70\u201390% Service Acceptance Rate (SAR) even with a 50% nodal outage for certain combinations of scenarios.<\/jats:p>","DOI":"10.3390\/fi13030082","type":"journal-article","created":{"date-parts":[[2021,3,23]],"date-time":"2021-03-23T14:08:16Z","timestamp":1616508496000},"page":"82","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Virtual Network Function Embedding under Nodal Outage Using Deep Q-Learning"],"prefix":"10.3390","volume":"13","author":[{"given":"Swarna Bindu","family":"Chetty","sequence":"first","affiliation":[{"name":"School of Electrical and Electronic Engineering, University College Dublin, Belfield, 4 Dublin, Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5508-8757","authenticated-orcid":false,"given":"Hamed","family":"Ahmadi","sequence":"additional","affiliation":[{"name":"Department of Electronic Engineering, University of York, Heslington, York YO10 5DD, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8358-2258","authenticated-orcid":false,"given":"Sachin","family":"Sharma","sequence":"additional","affiliation":[{"name":"School of Electrical and Electronic Engineering, Technological University Dublin, Grangegorman, 7 Dublin, Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1702-1492","authenticated-orcid":false,"given":"Avishek","family":"Nag","sequence":"additional","affiliation":[{"name":"School of Electrical and Electronic Engineering, University College Dublin, Belfield, 4 Dublin, Ireland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,3,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"134","DOI":"10.1109\/MNET.001.1900287","article-title":"A vision of 6G wireless systems: Applications, trends, technologies, and open research problems","volume":"34","author":"Saad","year":"2019","journal-title":"IEEE Netw."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1109\/COMST.2015.2477041","article-title":"Network function virtualization: State-of-the-art and research challenges","volume":"18","author":"Mijumbi","year":"2015","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1109\/MCOM.2019.1900271","article-title":"The roadmap to 6G: AI empowered wireless networks","volume":"57","author":"Letaief","year":"2019","journal-title":"IEEE Commun. Mag."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"518","DOI":"10.1109\/TNSM.2016.2598420","article-title":"Resource allocation in NFV: A comprehensive survey","volume":"13","author":"Herrera","year":"2016","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1318","DOI":"10.1109\/TNSM.2019.2947905","article-title":"A deep reinforcement learning approach for VNF Forwarding Graph Embedding","volume":"16","author":"Quang","year":"2019","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"542","DOI":"10.1109\/JSAC.2018.2815318","article-title":"vSPACE: VNF simultaneous placement, admission control and embedding","volume":"36","author":"Nejad","year":"2018","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1109\/TNSM.2018.2876623","article-title":"Single and multi-domain adaptive allocation algorithms for vnf forwarding graph embedding","volume":"16","author":"Quang","year":"2018","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Luizelli, M.C., Bays, L.R., Buriol, L.S., Barcellos, M.P., and Gaspary, L.P. (2015, January 11\u201315). Piecing together the NFV provisioning puzzle: Efficient placement and chaining of virtual network functions. Proceedings of the 2015 IFIP\/IEEE International Symposium on Integrated Network Management (IM), Ottawa, ON, Canada.","DOI":"10.1109\/INM.2015.7140281"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Chowdhury, S.R., Ahmed, R., Shahriar, N., Khan, A., Boutaba, R., Mitra, J., and Liu, L. (2017, January 8\u201312). Revine: Reallocation of virtual network embedding to eliminate substrate bottlenecks. Proceedings of the 2017 IFIP\/IEEE Symposium on Integrated Network and Service Management (IM), Lisbon, Portugal.","DOI":"10.23919\/INM.2017.7987271"},{"key":"ref_10","unstructured":"Xie, Y., Liu, Z., Wang, S., and Wang, Y. (2016). Service function chaining resource allocation: A survey. arXiv."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1029","DOI":"10.1109\/JSAC.2019.2906744","article-title":"DYVINE: Fitness-based dynamic virtual network embedding in cloud computing","volume":"37","author":"Dehury","year":"2019","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"433","DOI":"10.1109\/TNET.2018.2890631","article-title":"VNF placement and resource allocation for the support of vertical services in 5G networks","volume":"27","author":"Agarwal","year":"2019","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2532","DOI":"10.1109\/JSAC.2017.2760162","article-title":"Joint optimization of service function placement and flow distribution for service function chaining","volume":"35","author":"Jang","year":"2017","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Mijumbi, R., Serrat, J., Gorricho, J.L., Bouten, N., De Turck, F., and Davy, S. (2015, January 13\u201317). Design and evaluation of algorithms for mapping and scheduling of virtual network functions. Proceedings of the 2015 1st IEEE Conference on Network Softwarization (NetSoft), London, UK.","DOI":"10.1109\/NETSOFT.2015.7116120"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1995","DOI":"10.1007\/s00521-019-04376-6","article-title":"A Q-learning-based approach for virtual network embedding in data center","volume":"32","author":"Yuan","year":"2020","journal-title":"Neural Comput. Appl."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1292","DOI":"10.1109\/TNSM.2018.2867827","article-title":"z-TORCH: An automated NFV orchestration and monitoring solution","volume":"15","author":"Sciancalepore","year":"2018","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Chetty, S.B., Ahmadi, H., and Nag, A. (2020, January 14\u201317). Virtual Network Function Embedding under Nodal Outage using Reinforcement Learning. Proceedings of the IEEE International Conference on Advanced Networks and Telecommunications System, Delhi, India.","DOI":"10.1109\/ANTS50601.2020.9342803"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3133","DOI":"10.1109\/COMST.2019.2916583","article-title":"Applications of deep reinforcement learning in communications and networking: A survey","volume":"21","author":"Luong","year":"2019","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_19","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.comnet.2018.01.028","article-title":"On service-chaining strategies using virtual network functions in operator networks","volume":"133","author":"Gupta","year":"2018","journal-title":"Comput. Netw."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1765","DOI":"10.1109\/JSAC.2011.111002","article-title":"The Internet Topology Zoo","volume":"29","author":"Knight","year":"2011","journal-title":"Sel. Areas Commun. IEEE J."},{"key":"ref_23","first-page":"18","article-title":"On random graphs I","volume":"6","year":"1959","journal-title":"Publ. Math. Debr."},{"key":"ref_24","unstructured":"Chetty, S. (2021, March 04). VNFE Problem Using DQL Model. Available online: https:\/\/bitbucket.org\/swarnachetty\/deep-q-learning\/src\/master\/DQL."}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/13\/3\/82\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:39:41Z","timestamp":1760161181000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/13\/3\/82"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,23]]},"references-count":24,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2021,3]]}},"alternative-id":["fi13030082"],"URL":"https:\/\/doi.org\/10.3390\/fi13030082","relation":{},"ISSN":["1999-5903"],"issn-type":[{"type":"electronic","value":"1999-5903"}],"subject":[],"published":{"date-parts":[[2021,3,23]]}}}