{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,31]],"date-time":"2025-12-31T12:16:21Z","timestamp":1767183381048,"version":"3.41.0"},"reference-count":26,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2020,10,6]],"date-time":"2020-10-06T00:00:00Z","timestamp":1601942400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key R&D Program of China","doi-asserted-by":"crossref","award":["2018YFB1004800"],"award-info":[{"award-number":["2018YFB1004800"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Internet Technol."],"published-print":{"date-parts":[[2020,11,30]]},"abstract":"<jats:p>\n            Service migration is an often-used approach in cloud computing to minimize the access cost by moving the service close to most users. Although it is effective in a certain sense, the service migration in existing research still suffers from some deficiencies in its evolutionary abilities in\n            <jats:italic>scalability<\/jats:italic>\n            ,\n            <jats:italic>sensitivity<\/jats:italic>\n            , and\n            <jats:italic>adaptability<\/jats:italic>\n            to effectively react to the dynamically changing environments. This article proposes an evolutionary framework based on deep reinforcement learning for virtual service migration in large-scale mobile cloud centers. To enhance the spatio-temporal sensitivity of the algorithm, we design a scalable reward function for virtual service migration, redefine the input state, and add a Recurrent Neural Network (\n            <jats:italic>RNN<\/jats:italic>\n            ) to the learning framework. Additionally, in order to enhance the adaptability of the algorithm, we also decompose the action space and exploit the network cost to adjust the number of virtual machine (VMs). The experimental results show that, compared with the existing results, the migration strategy generated by the algorithm can not only significantly reduce the total service cost and achieve the load balancing at the same time, but also address the burst situations with low cost in dynamic environments.\n          <\/jats:p>","DOI":"10.1145\/3414840","type":"journal-article","created":{"date-parts":[[2020,10,6]],"date-time":"2020-10-06T11:49:08Z","timestamp":1601984948000},"page":"1-18","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["SMig-RL"],"prefix":"10.1145","volume":"20","author":[{"given":"Hongshuai","family":"Ren","sequence":"first","affiliation":[{"name":"Shenzhen Institutes of Advanced Technology, Shenzhen, Guangdong, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yang","family":"Wang","sequence":"additional","affiliation":[{"name":"Shenzhen Institutes of Advanced Technology, Shenzhen, Guangdong, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chengzhong","family":"Xu","sequence":"additional","affiliation":[{"name":"University of Macau, Macau, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xi","family":"Chen","sequence":"additional","affiliation":[{"name":"CAS Research Center for Ecology and Environment of Central Asia, Urumqi, Xingjiang, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2020,10,6]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/NAS.2017.8026876"},{"key":"e_1_2_1_2_1","volume-title":"A dynamic service-migration mechanism in edge cognitive computing.arXiv preprint arXiv:1808.07198","author":"Chen Min","year":"2018","unstructured":"Min Chen , Wei Li , Giancarlo Fortino , Yixue Hao , Long Hu , and Iztok Humar . 2018. A dynamic service-migration mechanism in edge cognitive computing.arXiv preprint arXiv:1808.07198 ( 2018 ). Min Chen, Wei Li, Giancarlo Fortino, Yixue Hao, Long Hu, and Iztok Humar. 2018. A dynamic service-migration mechanism in edge cognitive computing.arXiv preprint arXiv:1808.07198 (2018)."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/CEC.2015.7256960"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10586-017-0948-7"},{"key":"e_1_2_1_5_1","volume-title":"2005 International Conference on Parallel Processing (ICPP\u201905)","author":"Fu Song","year":"2005","unstructured":"Song Fu and Cheng Zhong Xu . 2005 . Service migration in distributed virtual machines for adaptive grid computing . In 2005 International Conference on Parallel Processing (ICPP\u201905) . 358--365. Song Fu and Cheng Zhong Xu. 2005. Service migration in distributed virtual machines for adaptive grid computing. In 2005 International Conference on Parallel Processing (ICPP\u201905). 358--365."},{"key":"e_1_2_1_6_1","doi-asserted-by":"crossref","unstructured":"Chaima Ghribi Makhlouf Hadji and Djamal Zeghlache. 2013. Energy efficient VM scheduling for cloud data centers: Exact allocation and migration algorithms. (2013) 671--678.  Chaima Ghribi Makhlouf Hadji and Djamal Zeghlache. 2013. Energy efficient VM scheduling for cloud data centers: Exact allocation and migration algorithms. (2013) 671--678.","DOI":"10.1109\/CCGrid.2013.89"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/CLOUD.2012.107"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-24797-2"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_2_1_10_1","first-page":"0","volume-title":"MA 1998, 322","author":"Johnson Jeffrey D.","year":"2000","unstructured":"Jeffrey D. Johnson , Jinghong Li , and Zengshi Chen . 2000 . Reinforcement learning: An introduction: R.S. Sutton, A.G. Barto, MIT Press, Cambridge , MA 1998, 322 pp. ISBN 0 - 262 -19398-1. Neurocomputing 35, 1 (2000), 205--206. Jeffrey D. Johnson, Jinghong Li, and Zengshi Chen. 2000. Reinforcement learning: An introduction: R.S. Sutton, A.G. Barto, MIT Press, Cambridge, MA 1998, 322 pp. ISBN 0-262-19398-1. Neurocomputing 35, 1 (2000), 205--206."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSAC.2016.2577238"},{"key":"e_1_2_1_12_1","volume-title":"International Conference on Learning Representations","author":"Diederik","year":"2015","unstructured":"Diederik P. Kingma and Jimmy Lei Ba. 2015. Adam: A method for stochastic optimization . International Conference on Learning Representations ( 2015 ). Diederik P. Kingma and Jimmy Lei Ba. 2015. Adam: A method for stochastic optimization. International Conference on Learning Representations (2015)."},{"key":"e_1_2_1_13_1","first-page":"A187","article-title":"Continuous control with deep reinforcement learning","volume":"8","author":"Lillicrap Timothy Paul","year":"2015","unstructured":"Timothy Paul Lillicrap , Jonathan James Hunt , Alexander Pritzel , Nicolas Manfred Otto Heess , Tom Erez , Yuval Tassa , David Silver , and Daniel Pieter Wierstra . 2015 . Continuous control with deep reinforcement learning . Computer Science 8 , 6 (2015), A187 . Timothy Paul Lillicrap, Jonathan James Hunt, Alexander Pritzel, Nicolas Manfred Otto Heess, Tom Erez, Yuval Tassa, David Silver, and Daniel Pieter Wierstra. 2015. Continuous control with deep reinforcement learning. Computer Science 8, 6 (2015), A187.","journal-title":"Computer Science"},{"key":"e_1_2_1_14_1","volume-title":"International Conference on Machine Learning (2016)","author":"Mnih Volodymyr","year":"2016","unstructured":"Volodymyr Mnih , Adri\u00e0 Puigdom\u00e8nech Badia , Mehdi Mirza , Alex Graves , Tim Harley , Timothy P. Lillicrap , David Silver , and Koray Kavukcuoglu . 2016 . Asynchronous methods for deep reinforcement learning . International Conference on Machine Learning (2016) , 1928--1937. Volodymyr Mnih, Adri\u00e0 Puigdom\u00e8nech Badia, Mehdi Mirza, Alex Graves, Tim Harley, Timothy P. Lillicrap, David Silver, and Koray Kavukcuoglu. 2016. Asynchronous methods for deep reinforcement learning. International Conference on Machine Learning (2016), 1928--1937."},{"key":"e_1_2_1_15_1","volume-title":"Human-level control through deep reinforcement learning. Nature 518, 7540","author":"Mnih Volodymyr","year":"2015","unstructured":"Volodymyr Mnih , Koray Kavukcuoglu , David Silver , Andrei A. Rusu , Joel Veness , Marc G. Bellemare , Alex Graves , Martin Riedmiller , Andreas K. Fidjeland , Georg Ostrovski , Stig Petersen , Charles Beattie , Amir Sadik , Ioannis Antonoglou , Helen King , Dharshan Kumaran , Daan Wierstra , Shane Legg , and Demis Hassabis . 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 ( 2015 ), 529--533. Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, and Demis Hassabis. 2015. Human-level control through deep reinforcement learning. Nature 518, 7540 (2015), 529--533."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSAC.2010.100109"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1555228.1555263"},{"volume-title":"2017 2nd International Conference on Computing and Communications Technologies (ICCCT). 55--58","author":"Ajay","key":"e_1_2_1_18_1","unstructured":"Ajay Thomas S. and Santhiya C . 2017. Dynamic resource scheduling using Delay time algorithm in Cloud environment . In 2017 2nd International Conference on Computing and Communications Technologies (ICCCT). 55--58 . Ajay Thomas S. and Santhiya C. 2017. Dynamic resource scheduling using Delay time algorithm in Cloud environment. In 2017 2nd International Conference on Computing and Communications Technologies (ICCCT). 55--58."},{"key":"e_1_2_1_19_1","volume-title":"International Conference on Machine Learning (2015)","author":"Schulman John","year":"2015","unstructured":"John Schulman , Sergey Levine , Pieter Abbeel , Michael I. Jordan , and Philipp Moritz . 2015 . Trust region policy optimization . International Conference on Machine Learning (2015) , 1889--1897. John Schulman, Sergey Levine, Pieter Abbeel, Michael I. Jordan, and Philipp Moritz. 2015. Trust region policy optimization. International Conference on Machine Learning (2015), 1889--1897."},{"key":"e_1_2_1_20_1","volume-title":"Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347","author":"Schulman John","year":"2017","unstructured":"John Schulman , Filip Wolski , Prafulla Dhariwal , Alec Radford , and Oleg Klimov . 2017. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347 ( 2017 ). John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347 (2017)."},{"volume-title":"Proceedings of the 31st International Conference on Machine Learning. 387--395","author":"Silver David","key":"e_1_2_1_21_1","unstructured":"David Silver , Guy Lever , Nicolas Heess , Thomas Degris , Daan Wierstra , and Martin A. Riedmiller . 2014. Deterministic policy gradient algorithms . In Proceedings of the 31st International Conference on Machine Learning. 387--395 . David Silver, Guy Lever, Nicolas Heess, Thomas Degris, Daan Wierstra, and Martin A. Riedmiller. 2014. Deterministic policy gradient algorithms. In Proceedings of the 31st International Conference on Machine Learning. 387--395."},{"key":"e_1_2_1_22_1","unstructured":"Richard S. Sutton David A. McAllester Satinder P. Singh and Yishay Mansour. 1999. Policy gradient methods for reinforcement learning with function approximation. In Advances in Neural Information Processing Systems 12. 1057--1063.  Richard S. Sutton David A. McAllester Satinder P. Singh and Yishay Mansour. 1999. Policy gradient methods for reinforcement learning with function approximation. In Advances in Neural Information Processing Systems 12. 1057--1063."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.peva.2015.06.013"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/5.58337"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jpdc.2011.10.003"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICAC.2017.47"}],"container-title":["ACM Transactions on Internet Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3414840","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3414840","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:23:45Z","timestamp":1750202625000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3414840"}},"subtitle":["An Evolutionary Migration Framework for Cloud Services Based on Deep Reinforcement Learning"],"short-title":[],"issued":{"date-parts":[[2020,10,6]]},"references-count":26,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2020,11,30]]}},"alternative-id":["10.1145\/3414840"],"URL":"https:\/\/doi.org\/10.1145\/3414840","relation":{},"ISSN":["1533-5399","1557-6051"],"issn-type":[{"type":"print","value":"1533-5399"},{"type":"electronic","value":"1557-6051"}],"subject":[],"published":{"date-parts":[[2020,10,6]]},"assertion":[{"value":"2019-07-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-07-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-10-06","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}