{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,25]],"date-time":"2026-02-25T18:01:52Z","timestamp":1772042512413,"version":"3.50.1"},"reference-count":40,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2018,11,16]],"date-time":"2018-11-16T00:00:00Z","timestamp":1542326400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>As the size and service requirements of today\u2019s networks gradually increase, large numbers of proprietary devices are deployed, which leads to network complexity, information security crises and makes network service and service provider management increasingly difficult. Network function virtualization (NFV) technology is one solution to this problem. NFV separates network functions from hardware and deploys them as software on a common server. NFV can be used to improve service flexibility and isolate the services provided for each user, thus guaranteeing the security of user data. Therefore, the use of NFV technology includes many problems worth studying. For example, when there is a free choice of network path, one problem is how to choose a service function chain (SFC) that both meets the requirements and offers the service provider maximum profit. Most existing solutions are heuristic algorithms with high time efficiency, or integer linear programming (ILP) algorithms with high accuracy. It\u2019s necessary to design an algorithm that symmetrically considers both time efficiency and accuracy. In this paper, we propose the Q-learning Framework Hybrid Module algorithm (QLFHM), which includes reinforcement learning to solve this SFC deployment problem in dynamic networks. The reinforcement learning module in QLFHM is responsible for the output of alternative paths, while the load balancing module in QLFHM is responsible for picking the optimal solution from them. The results of a comparison simulation experiment on a dynamic network topology show that the proposed algorithm can output the approximate optimal solution in a relatively short time while also considering the network load balance. Thus, it achieves the goal of maximizing the benefit to the service provider.<\/jats:p>","DOI":"10.3390\/sym10110646","type":"journal-article","created":{"date-parts":[[2018,11,16]],"date-time":"2018-11-16T11:48:31Z","timestamp":1542368911000},"page":"646","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":32,"title":["A Q-Learning-Based Approach for Deploying Dynamic Service Function Chains"],"prefix":"10.3390","volume":"10","author":[{"given":"Jian","family":"Sun","sequence":"first","affiliation":[{"name":"Key Lab of Optical Fiber Sensing and Communications (Ministry of Education), University of Electronic Science and Technology of China, Chengdu 611731, China"}],"role":[{"role":"author","vocab":"crossref"}]},{"given":"Guanhua","family":"Huang","sequence":"additional","affiliation":[{"name":"Key Lab of Optical Fiber Sensing and Communications (Ministry of Education), University of Electronic Science and Technology of China, Chengdu 611731, China"}],"role":[{"role":"author","vocab":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2448-8915","authenticated-orcid":false,"given":"Gang","family":"Sun","sequence":"additional","affiliation":[{"name":"Key Lab of Optical Fiber Sensing and Communications (Ministry of Education), University of Electronic Science and Technology of China, Chengdu 611731, China"},{"name":"Center for Cyber Security, University of Electronic Science and Technology of China, Chengdu 611731, China"}],"role":[{"role":"author","vocab":"crossref"}]},{"given":"Hongfang","family":"Yu","sequence":"additional","affiliation":[{"name":"Key Lab of Optical Fiber Sensing and Communications (Ministry of Education), University of Electronic Science and Technology of China, Chengdu 611731, China"},{"name":"Center for Cyber Security, University of Electronic Science and Technology of China, Chengdu 611731, China"}],"role":[{"role":"author","vocab":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0229-2460","authenticated-orcid":false,"given":"Arun Kumar","family":"Sangaiah","sequence":"additional","affiliation":[{"name":"School of Computing Science and Engineering, Vellore Institute of Technology, Tamil Nadu 632014, India"}],"role":[{"role":"author","vocab":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8012-5852","authenticated-orcid":false,"given":"Victor","family":"Chang","sequence":"additional","affiliation":[{"name":"International Business School Suzhou (IBSS), Xi\u2019an Jiaotong-Liverpool University, Suzhou 215123, China"}],"role":[{"role":"author","vocab":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,11,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1368","DOI":"10.1016\/j.future.2018.05.022","article-title":"Big Data and Internet of Things\u2014Fusion for different services and its impacts","volume":"86","author":"Sun","year":"2018","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1016\/j.jnca.2018.06.015","article-title":"AmoebaNet: An SDN-enabled network service for big data science","volume":"119","author":"Shah","year":"2018","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1109\/JSYST.2013.2289584","article-title":"Power-efficient provisioning for online virtual network requests in cloud-based data centers","volume":"9","author":"Sun","year":"2015","journal-title":"IEEE Syst. J."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"684","DOI":"10.1109\/LCOMM.2016.2526001","article-title":"Distributed online scheduling and routing of multicast-oriented tasks for profit-driven cloud computing","volume":"20","author":"Wu","year":"2016","journal-title":"IEEE Commun. Lett."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"457","DOI":"10.1016\/j.future.2017.09.065","article-title":"Towards Provisioning Hybrid Virtual Networks in Federated Cloud Data Centers","volume":"87","author":"Sun","year":"2018","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"518","DOI":"10.1109\/TNSM.2016.2598420","article-title":"Resource Allocation in NFV: A Comprehensive Survey","volume":"13","author":"Herrera","year":"2017","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"212","DOI":"10.1016\/j.comnet.2018.01.021","article-title":"A comprehensive survey of Network Function Virtualization","volume":"133","author":"Yi","year":"2018","journal-title":"Comput. Netw."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1109\/COMST.2015.2477041","article-title":"Network Function Virtualization: State-of-the-Art and Research Challenges","volume":"18","author":"Mijumbi","year":"2016","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1016\/j.ins.2017.08.021","article-title":"The Cost-efficient Deployment of Replica Servers in Virtual Content Distribution Networks for Data Fusion","volume":"432","author":"Sun","year":"2018","journal-title":"Inf. Sci."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1539","DOI":"10.1109\/LCOMM.2016.2580151","article-title":"Joint spectrum and IT resource allocation for efficient vNF service chaining in inter-datacenter elastic optical networks","volume":"20","author":"Fang","year":"2016","journal-title":"IEEE Commun. Lett."},{"key":"ref_11","first-page":"543","article-title":"Service Function Chaining (SFC) Operation, Administration and Maintenance (OAM) Framework","volume":"90","author":"Ghanwani","year":"2017","journal-title":"J. Am. Chem. Soc."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1364\/JOCN.7.000314","article-title":"Joint defragmentation of optical spectrum and IT resources in elastic optical datacenter interconnections","volume":"7","author":"Fang","year":"2015","journal-title":"J. Opt. Commun. Netw."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"711","DOI":"10.1109\/TNSM.2016.2580668","article-title":"Customizable function chains: Managing service chain variability in hybrid NFV networks","volume":"13","author":"Moens","year":"2016","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"543","DOI":"10.1109\/TNSM.2017.2711610","article-title":"On dynamic service function chain deployment and readjustment","volume":"14","author":"Liu","year":"2017","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Mars, P., and Chen, J.R. (2018). Learning Algorithms: Theory and Applications in Signal Processing, Control and Communications, CRC Press.","DOI":"10.1201\/9781351073974"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Apostolopoulos, P.A., Tsiropoulou, E.E., and Papavassiliou, S. (2018). Demand Response Management in Smart Grid Networks: A Two-Stage Game-Theoretic Learning-Based Approach. Mob. Netw. Appl., 1\u201314.","DOI":"10.1007\/s11036-018-1124-x"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Tsiropoulou, E.E., Katsinis, G.K., and Filios, A. (2014, January 22\u201327). On the Problem of Optimal Cell Selection and Uplink Power Control in Open Access Multi-service Two-Tier Femtocell Networks. Proceedings of the International Conference on Ad-Hoc Networks and Wireless, Benidorm, Spain.","DOI":"10.1007\/978-3-319-07425-2_9"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"538","DOI":"10.1016\/j.apenergy.2017.11.072","article-title":"Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicle","volume":"211","author":"Xiong","year":"2018","journal-title":"Appl. Energy"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1016\/j.neucom.2017.08.036","article-title":"Data-driven model-free slip control of anti-lock braking systems using reinforcement Q-learning","volume":"275","author":"Radac","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"4087","DOI":"10.1109\/TVT.2018.2789466","article-title":"UAV Relay in VANETs Against Smart Jamming with Reinforcement Learning","volume":"67","author":"Xiao","year":"2018","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1109\/3468.736368","article-title":"Multiple stochastic learning automata for vehicle path control in an automated highway system","volume":"29","author":"Unsal","year":"2002","journal-title":"IEEE Trans. Syst. Man Cybern. Part A Syst. Hum."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Barto, A.G., Anandan, P., and Anderson, C.W. (1986). Cooperativity in networks of pattern recognizing stochastic learning automata. Adaptive and Learning Systems, Springer.","DOI":"10.1007\/978-1-4757-1895-9_16"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"765","DOI":"10.1007\/978-981-10-8672-4_58","article-title":"Occupancy overload control by Q-learning","volume":"480","author":"Khazaei","year":"2019","journal-title":"Lect. Notes Electr. Eng."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1109\/MSP.2017.2743240","article-title":"Deep Reinforcement Learning A brief survey","volume":"34","author":"Kai","year":"2017","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"775","DOI":"10.1016\/j.neuroimage.2018.07.043","article-title":"Generative adversarial networks for reconstructing natural images from brain activity","volume":"181","author":"Seeliger","year":"2018","journal-title":"Neuroimage"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1016\/j.future.2016.12.019","article-title":"The Efficient Framework and Algorithm for Provisioning Evolving VDC in Federated Data Centers","volume":"73","author":"Sun","year":"2017","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1016\/j.future.2015.09.005","article-title":"A New Technique for Efficient Live Migration of Multiple Virtual Machines","volume":"55","author":"Sun","year":"2016","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"725","DOI":"10.1109\/TNSM.2016.2569020","article-title":"Orchestrating virtualized network functions","volume":"13","author":"Bari","year":"2016","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Li, D., Lan, J.L., and Wang, P. (2018). Joint service function chain deploying and path selection for bandwidth saving and VNF reuse. Int. J. Commun. Syst., 31.","DOI":"10.1002\/dac.3523"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1109\/TSC.2015.2477825","article-title":"Live Migration for Multiple Correlated Virtual Machines in Cloud-based Data Centers","volume":"11","author":"Sun","year":"2018","journal-title":"IEEE Trans. Serv. Comput."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Luizelli, M.C., Bays, L.R., and Buriol, L.S. (2015, January 11\u201315). Piecing together the NFV provisioning puzzle: Efficient placement and chaining of virtual network functions. Proceedings of the IFIP\/IEEE International Symposium on Integrated Network Management (IM), Ottawa, ON, Canada.","DOI":"10.1109\/INM.2015.7140281"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1175","DOI":"10.1109\/TNSM.2018.2861717","article-title":"Service Function Chain Orchestration across Multiple domains: A Full Mesh Aggregation Approach","volume":"15","author":"Sun","year":"2018","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_33","unstructured":"Gupta, A., Habib, M.F., and Chowdhury, P. (2015). Joint Virtual Network Function Placement and Routing of Traffic in Operator Network, University of California Davis. Technical Report."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1016\/j.future.2018.09.037","article-title":"Energy-efficient and Traffic-aware Service Function Chaining Orchestration in Multi-Domain Networks","volume":"91","author":"Sun","year":"2019","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1016\/j.future.2018.03.018","article-title":"Low-Latency Orchestration for Workflow-Oriented Service Function Chain in Edge Computing","volume":"85","author":"Sun","year":"2018","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Kim, S.I., and Kim, H.S. (2017, January 4\u20137). A research on dynamic service function chaining based on reinforcement learning using resource usage. Proceedings of the International Conference on Ubiquitous & Future Networks, Milan, Italy.","DOI":"10.1109\/ICUFN.2017.7993856"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1319","DOI":"10.1016\/j.future.2013.02.002","article-title":"Two levels autonomic resource management in virtualized IaaS","volume":"29","author":"Tchana","year":"2013","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.future.2015.05.013","article-title":"Enforcing CPU allocation in a heterogeneous IaaS","volume":"53","author":"Teabe","year":"2015","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.future.2015.11.027","article-title":"Software consolidation as an efficient energy and cost saving solution","volume":"58","author":"Tchana","year":"2016","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/j.future.2013.12.037","article-title":"Coordinating self-sizing and self-repair managers for multi-tier systems","volume":"35","author":"Gueye","year":"2014","journal-title":"Future Gener. Comput.Syst."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/10\/11\/646\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:30:15Z","timestamp":1760196615000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/10\/11\/646"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,11,16]]},"references-count":40,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2018,11]]}},"alternative-id":["sym10110646"],"URL":"https:\/\/doi.org\/10.3390\/sym10110646","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,11,16]]}}}