{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T15:49:16Z","timestamp":1782834556695,"version":"3.54.5"},"reference-count":16,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2024,9,25]],"date-time":"2024-09-25T00:00:00Z","timestamp":1727222400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key R&amp;D Program of China","award":["2020YFB1806702"],"award-info":[{"award-number":["2020YFB1806702"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Future 6G networks will inherit and develop Network Function Virtualization (NFV) architecture. With the NFV-enabled network architecture, it becomes possible to establish different virtual networks within the same infrastructure, create different Virtual Network Functions (VNFs) in different virtual networks, and form Service Function Chains (SFCs) that meet different service requirements through the orderly combination of VNFs. These SFCs can be deployed to physical entities as needed to provide network functions that support different services. To meet the highly dynamic service requirements in the future 6G Internet of Things (IoT) scenario, the highly flexible and efficient SFC reconfiguration algorithm is the key research direction. Deep-learning-based algorithms have shown their advantages in solving this type of dynamic optimization problem. Considering that the efficiency of the traditional Actor Critic (AC) algorithm is limited, the policy does not directly participate in the value function update. In this paper, we use the Proximal Policy Optimization (PPO) clip function to restrict the difference between the new policy and the old policy, to ensure the stability of the updating process. We combine PPO with AC, and further bring the historical decision information as the network knowledge to offer better initial policies, to accelerate the training speed. We also propose the Knowledge = Assisted Actor Critic Proximal Policy Optimization (KA-ACPPO)-based SFC reconfiguration algorithm to ensure the Quality of Service (QoS) of end-to-end services. Simulation results show that the proposed KA-ACPPO algorithm can effectively reduce computing cost and power consumption.<\/jats:p>","DOI":"10.3390\/e26100820","type":"journal-article","created":{"date-parts":[[2024,9,26]],"date-time":"2024-09-26T08:20:52Z","timestamp":1727338852000},"page":"820","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Knowledge-Assisted Actor Critic Proximal Policy Optimization-Based Service Function Chain Reconfiguration Algorithm for 6G IoT Scenario"],"prefix":"10.3390","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2855-9714","authenticated-orcid":false,"given":"Bei","family":"Liu","sequence":"first","affiliation":[{"name":"School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shuting","family":"Long","sequence":"additional","affiliation":[{"name":"School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xin","family":"Su","sequence":"additional","affiliation":[{"name":"Department of Electronic Engineering, Tsinghua University, Beijing 100084, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,9,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1109\/MCOM.001.1900411","article-title":"Towards 6G Networks: Use Cases and Technologies","volume":"35","author":"Giordani","year":"2020","journal-title":"IEEE Commun. Mag."},{"key":"ref_2","first-page":"6","article-title":"6G-ADM: Knowledge based 6G network management and control architecture","volume":"43","author":"Liao","year":"2022","journal-title":"J. Commun."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Wen, T., Yu, H., Sun, G., and Liu, L. (2016, January 22\u201327). Network function consolidation in service function chaining orchestration. Proceedings of the 2016 IEEE International Conference on Communications (ICC), Kuala Lumpur, Malaysia.","DOI":"10.1109\/ICC.2016.7510679"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"979","DOI":"10.1109\/TNSM.2023.3287757","article-title":"Cost-Efficient Cluster Migration of VNFs for Service Function Chain Embedding","volume":"21","author":"Afrasiabi","year":"2024","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Tran, V., Sun, J., Tang, B., and Pan, D. (June, January 30). Traffic-Optimal Virtual Network Function Placement and Migration in Dynamic Cloud Data Centers. Proceedings of the 2022 IEEE International Parallel and Distributed Processing Symposium (IPDPS), Lyon, France.","DOI":"10.1109\/IPDPS53621.2022.00094"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"979","DOI":"10.1109\/TNET.2021.3055074","article-title":"Prioritized Deployment of Dynamic Service Function Chains","volume":"29","author":"Farkiani","year":"2021","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Zhang, F., Lu, H., Guo, F., and Gu, Z. (2021, January 7\u201311). Traffic Prediction Based VNF Migration with Temporal Convolutional Network. Proceedings of the 2021 IEEE Global Communications Conference (GLOBECOM), Madrid, Spain.","DOI":"10.1109\/GLOBECOM46510.2021.9685818"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1866","DOI":"10.1109\/TNSM.2022.3217723","article-title":"Reinforcement Learning-Based Optimization Framework for Application Component Migration in NFV Cloud-Fog Environments","volume":"20","author":"Afrasiabi","year":"2023","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Ibrahimpasic, A.L., Han, B., and Schotten, H.D. (2021, January 29). AI-Empowered VNF Migration as a Cost-Loss-Effective Solution for Network Resilience. Proceedings of the 2021 IEEE Wireless Communications and Networking Conference Workshops (WCNCW), Nanjing, China.","DOI":"10.1109\/WCNCW49093.2021.9420029"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Liu, Y., Zhang, C., Yang, H., Zhang, S., Wang, X., and Li, F. (2023, January 21\u201323). A Deep Reinforcement Learning-Based Approach for Adaptive SFC Deployment in Multi-Domain Networks. Proceedings of the 2023 15th International Conference on Communication Software and Networks (ICCSN), Shenyang, China.","DOI":"10.1109\/ICCSN57992.2023.10297346"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Hirayama, T., and Jibiki, M. (2022, January 28\u201330). SFC Path Selection based on Combination of Topological Analysis and Demand Prediction. Proceedings of the 2022 23rd Asia-Pacific Network Operations and Management Symposium (APNOMS), Takamatsu, Japan.","DOI":"10.23919\/APNOMS56106.2022.9919998"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Wu, Y., Jia, Z., Wu, Q., and Lu, Z. (2024). Adaptive QoE-Aware SFC Orchestration in UAV Networks: A Deep Reinforcement Learning Approach. IEEE Trans. Netw. Sci. Eng.","DOI":"10.1109\/TNSE.2024.3442857"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Liu, N., Li, Z., Xu, J., Xu, Z., Lin, S., Qiu, Q., Tang, J., and Wang, Y. (2017, January 5\u20138). A hierarchical framework of cloud resource allocation and power management using deep reinforcement learning. Proceedings of the IEEE 37th International Conference on Distributed Computing Systems, Atlanta, GA, USA.","DOI":"10.1109\/ICDCS.2017.123"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Luo, X., Liu, T., Xie, S., Wang, J., Wang, W., Li, Y., and Peng, Y. (2019, January 4\u20136). Proximal Policy Optimization with Mixed Distributed Training. Proceedings of the 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI), Portland, OR, USA.","DOI":"10.1109\/ICTAI.2019.00206"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1109\/TWC.2019.2946797","article-title":"Dynamic Service Function Chain Embedding for NFV-Enabled IoT: A Deep Reinforcement Learning Approach","volume":"19","author":"Fu","year":"2020","journal-title":"IEEE Trans. Wirel. Commun."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Yao, J., and Chen, M. (2020, January 11\u201314). A Flexible Deployment Scheme for Virtual Network Function Based on Reinforcement Learning. Proceedings of the 2020 IEEE 6th International Conference on Computer and Communications (ICCC), Chengdu, China.","DOI":"10.1109\/ICCC51575.2020.9344881"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/10\/820\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:03:13Z","timestamp":1760112193000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/10\/820"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,25]]},"references-count":16,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2024,10]]}},"alternative-id":["e26100820"],"URL":"https:\/\/doi.org\/10.3390\/e26100820","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,25]]}}}