{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,18]],"date-time":"2026-06-18T15:43:45Z","timestamp":1781797425151,"version":"3.54.5"},"reference-count":68,"publisher":"Springer Science and Business Media LLC","issue":"19","license":[{"start":{"date-parts":[[2020,12,12]],"date-time":"2020-12-12T00:00:00Z","timestamp":1607731200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,12,12]],"date-time":"2020-12-12T00:00:00Z","timestamp":1607731200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100004702","name":"Universit\u00e0 degli Studi di Genova","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004702","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Soft Comput"],"published-print":{"date-parts":[[2021,10]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The ubiquitous diffusion of cloud computing requires suitable management policies to face the workload while guaranteeing quality constraints and mitigating costs. The typical trade-off is between the used power and the adherence to a service-level metric subscribed by customers. To this aim, a possible idea is to use an optimization-based placement mechanism to select the servers where to deploy virtual machines. Unfortunately, high packing factors could lead to performance and security issues, e.g., virtual machines can compete for hardware resources or collude to leak data. Therefore, we introduce a multi-objective approach to compute optimal placement strategies considering different goals, such as the impact of hardware outages, the power required by the datacenter, and the performance perceived by users. Placement strategies are found by using a deep reinforcement learning framework to select the best placement heuristic for each virtual machine composing the workload. Results indicate that our method outperforms bin packing heuristics widely used in the literature when considering either synthetic or real workloads.<\/jats:p>","DOI":"10.1007\/s00500-020-05462-x","type":"journal-article","created":{"date-parts":[[2020,12,12]],"date-time":"2020-12-12T10:02:36Z","timestamp":1607767356000},"page":"12569-12588","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":66,"title":["Deep reinforcement learning for multi-objective placement of virtual machines in cloud datacenters"],"prefix":"10.1007","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6466-3354","authenticated-orcid":false,"given":"Luca","family":"Caviglione","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5048-4141","authenticated-orcid":false,"given":"Mauro","family":"Gaggero","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0618-8406","authenticated-orcid":false,"given":"Massimo","family":"Paolucci","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6846-6374","authenticated-orcid":false,"given":"Roberto","family":"Ronco","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2020,12,12]]},"reference":[{"key":"5462_CR1","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1016\/j.jnca.2015.02.002","volume":"52","author":"RW Ahmad","year":"2015","unstructured":"Ahmad RW, Gani A, Hamid SHA, Shiraz M, Yousafzai A, Xia F (2015) A survey on virtual machine migration and server consolidation frameworks for cloud data centers. J Netw Comput Appl 52:11\u201325","journal-title":"J Netw Comput Appl"},{"key":"5462_CR2","doi-asserted-by":"crossref","unstructured":"Arulkumaran K, Deisenroth MP, Brundage M, Bharath AA (2017) A brief survey of deep reinforcement learning. arXiv:1708.05866","DOI":"10.1109\/MSP.2017.2743240"},{"key":"5462_CR3","doi-asserted-by":"crossref","unstructured":"Beloglazov A, Buyya R (2010) Adaptive threshold-based approach for energy-efficient consolidation of virtual machines in cloud data centers. In: Proceedings of international workshop on middleware for grids, clouds and e-Science, pp 1\u20134","DOI":"10.1145\/1890799.1890803"},{"key":"5462_CR4","doi-asserted-by":"crossref","unstructured":"Bobroff N, Kochut A, Beaty K (2007) Dynamic placement of virtual machines for managing SLA violations. In: International symposium on integrated network management, pp 119\u2013128","DOI":"10.1109\/INM.2007.374776"},{"key":"5462_CR5","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TCIAIG.2012.2186810","volume":"4","author":"C Browne","year":"2012","unstructured":"Browne C, Powley EJ, Whitehouse D, Lucas SM, Cowling PI, Rohlfshagen P, Tavener S, Liebana DP, Samothrakis S, Colton S (2012) A survey of monte carlo tree search methods. IEEE Trans Comput Intell AI Games 4:1\u201343","journal-title":"IEEE Trans Comput Intell AI Games"},{"key":"5462_CR6","doi-asserted-by":"crossref","unstructured":"Caron E, Cornabas JR (2014) Improving users\u2019 isolation in IaaS: Virtual machine placement with security constraints. In: International conference on cloud computing, pp 64\u201371","DOI":"10.1109\/CLOUD.2014.19"},{"issue":"7","key":"5462_CR7","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1109\/MCOM.2017.1600955","volume":"55","author":"L Caviglione","year":"2017","unstructured":"Caviglione L, Gaggero M, Cambiaso E, Aiello M (2017) Measuring the energy consumption of cyber security. IEEE Commun Mag 55(7):58\u201363","journal-title":"IEEE Commun Mag"},{"issue":"2","key":"5462_CR8","doi-asserted-by":"publisher","first-page":"1298","DOI":"10.1109\/TII.2020.2978771","volume":"17","author":"L Caviglione","year":"2021","unstructured":"Caviglione L, Gaggero M (2021) Multiobjective placement for secure and dependable smart industrial environments. IEEE Trans Ind Inform 17(2):1298\u20131306","journal-title":"IEEE Trans Ind Inform"},{"key":"5462_CR9","doi-asserted-by":"crossref","unstructured":"Chen X, Zhang H, Wu C, Mao S, Ji Y, Bennis M (2018) Performance optimization in mobile-edge computing via deep reinforcement learning. In: IEEE vehicular technology conference, pp 1\u20136","DOI":"10.1109\/VTCFall.2018.8690980"},{"key":"5462_CR10","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1002\/dac.4068","volume":"32","author":"N Donyagard Vahed","year":"2019","unstructured":"Donyagard Vahed N, Ghobaei-Arani M, Souri A (2019) Multiobjective virtual machine placement mechanisms using nature-inspired metaheuristic algorithms in cloud environments: a comprehensive review. Int J Commun Syst 32:1\u201332","journal-title":"Int J Commun Syst"},{"key":"5462_CR11","doi-asserted-by":"crossref","unstructured":"Duan Y, Fu G, Zhou N, Sun X, Narendra NC, Hu B (2015) Everything as a service (XaaS) on the cloud: origins, current and future trends. In: International conference on cloud computing, pp 621\u2013628","DOI":"10.1109\/CLOUD.2015.88"},{"key":"5462_CR12","doi-asserted-by":"crossref","unstructured":"Farahnakian F, Liljeberg P, Plosila J (2014) Energy-efficient virtual machines consolidation in cloud data centers using reinforcement learning. In: Euromicro international conference on parallel, distributed, and network-based processing, pp 500\u2013507","DOI":"10.1109\/PDP.2014.109"},{"key":"5462_CR13","doi-asserted-by":"crossref","unstructured":"Farzai S, Shirvani MH, Rabbani M (2020) Multi-objective communication-aware optimization for virtual machine placement in cloud datacenters. Sustain Comput Inform Syst, art. no. 100374","DOI":"10.1016\/j.suscom.2020.100374"},{"key":"5462_CR14","doi-asserted-by":"crossref","unstructured":"Ferdaus MH, Murshed M, Calheiros RN, Buyya R (2014) Virtual machine consolidation in cloud data centers using ACO metaheuristic. In: European conference on parallel processing, pp 306\u2013317","DOI":"10.1007\/978-3-319-09873-9_26"},{"key":"5462_CR15","unstructured":"Fortunato M, Azar MG, Piot B, Menick J, Osband I, Graves A, Mnih V, Munos R, Hassabis D, Pietquin O, Blundell C, Legg S (2018) Noisy networks for exploration. In: Proceedings of the international conference on representation learning (ICLR 2018), Vancouver (Canada)"},{"issue":"200","key":"5462_CR16","doi-asserted-by":"publisher","first-page":"675","DOI":"10.1080\/01621459.1937.10503522","volume":"32","author":"M Friedman","year":"1937","unstructured":"Friedman M (1937) The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J Am Stat Assoc 32(200):675\u2013701","journal-title":"J Am Stat Assoc"},{"issue":"2","key":"5462_CR17","first-page":"461","volume":"24","author":"M Gaggero","year":"2016","unstructured":"Gaggero M, Caviglione L (2016) Predictive control for energy-aware consolidation in cloud datacenters. IEEE Trans Contr Syst Technol 24(2):461\u2013474","journal-title":"IEEE Trans Contr Syst Technol"},{"issue":"1","key":"5462_CR18","doi-asserted-by":"publisher","first-page":"420","DOI":"10.1109\/TASE.2018.2826723","volume":"16","author":"M Gaggero","year":"2019","unstructured":"Gaggero M, Caviglione L (2019) Model predictive control for energy-efficient, quality-aware, and secure virtual machine placement. IEEE Trans Autom Sci Eng 16(1):420\u2013432","journal-title":"IEEE Trans Autom Sci Eng"},{"issue":"8","key":"5462_CR19","doi-asserted-by":"publisher","first-page":"1230","DOI":"10.1016\/j.jcss.2013.02.004","volume":"79","author":"Y Gao","year":"2013","unstructured":"Gao Y, Guan H, Qi Z, Hou Y, Liu L (2013) A multi-objective ant colony system algorithm for virtual machine placement in cloud computing. J Comput Syst Sci 79(8):1230\u20131242","journal-title":"J Comput Syst Sci"},{"issue":"6","key":"5462_CR20","doi-asserted-by":"publisher","first-page":"1149","DOI":"10.1080\/0952813X.2017.1310308","volume":"29","author":"M Ghobaei-Arani","year":"2017","unstructured":"Ghobaei-Arani M, Shamsi M, Rahmanian AA (2017) An efficient approach for improving virtual machine placement in cloud computing environment. J Exp Theor Artif Intell 29(6):1149\u20131171","journal-title":"J Exp Theor Artif Intell"},{"key":"5462_CR21","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1002\/dac.3537","volume":"31","author":"M Ghobaei-Arani","year":"2018","unstructured":"Ghobaei-Arani M, Rahmanian AA, Shamsi M, Rasouli-Kenari A (2018) A learning-based approach for virtual machine placement in cloud data centers. Int J Commun Syst 31:1\u201318","journal-title":"Int J Commun Syst"},{"key":"5462_CR22","doi-asserted-by":"publisher","first-page":"106912","DOI":"10.1109\/ACCESS.2019.2932462","volume":"7","author":"M Ghobaei-Arani","year":"2019","unstructured":"Ghobaei-Arani M, Souri A, Baker T, Hussien A (2019) Controcity: an autonomous approach for controlling elasticity using buffer management in cloud computing environment. IEEE Access 7:106912\u2013106924","journal-title":"IEEE Access"},{"key":"5462_CR23","doi-asserted-by":"crossref","unstructured":"Guenter B, Jain N, Williams C (2011) Managing cost, performance, and reliability tradeoffs for energy-aware server provisioning. In: Proceedings of IEEE INFOCOM, pp 1332\u20131340","DOI":"10.1109\/INFCOM.2011.5934917"},{"key":"5462_CR24","doi-asserted-by":"crossref","unstructured":"Hessel M, Modayil J, van Hasselt H, Schaul T, Ostrovski G, Dabney W, Horgan D, Piot B, Azar M, Silver D (2018) Rainbow: combining improvements in deep reinforcement learning. In: 32nd AAAI conference on artificial intelligence","DOI":"10.1609\/aaai.v32i1.11796"},{"issue":"1","key":"5462_CR25","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1186\/s13677-018-0111-x","volume":"7","author":"S Ismaeel","year":"2018","unstructured":"Ismaeel S, Karim R, Miri A (2018) Proactive dynamic virtual-machine consolidation for energy conservation in cloud data centres. J Cloud Comput 7(1):10","journal-title":"J Cloud Comput"},{"key":"5462_CR26","unstructured":"Ivanov S, D\u2019yakonov A (2019) Modern deep reinforcement learning algorithms. arXiv:1906.10025"},{"key":"5462_CR27","doi-asserted-by":"crossref","unstructured":"Jhawar R, Piuri V, Samarati P (2012) Supporting security requirements for resource management in cloud computing. In: International conference computational science and engineering, pp 170\u2013177","DOI":"10.1109\/ICCSE.2012.32"},{"key":"5462_CR28","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1613\/jair.301","volume":"4","author":"LP Kaelbling","year":"1996","unstructured":"Kaelbling LP, Littman ML, Moore AW (1996) Reinforcement learning: a survey. J Artif Intell Res 4:237\u2013285","journal-title":"J Artif Intell Res"},{"issue":"2","key":"5462_CR29","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/2742488","volume":"48","author":"T Kaur","year":"2015","unstructured":"Kaur T, Chana I (2015) Energy efficiency techniques in cloud computing: a survey and taxonomy. ACM Comput Surv 48(2):1\u201346","journal-title":"ACM Comput Surv"},{"key":"5462_CR30","unstructured":"Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. arXiv:1412.6980"},{"key":"5462_CR31","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1016\/j.future.2017.10.047","volume":"81","author":"J Kumar","year":"2018","unstructured":"Kumar J, Singh AK (2018) Workload prediction in cloud using artificial neural network and adaptive differential evolution. Future Gener Comput Syst 81:41\u201352","journal-title":"Future Gener Comput Syst"},{"issue":"1","key":"5462_CR32","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10586-008-0070-y","volume":"12","author":"D Kusic","year":"2009","unstructured":"Kusic D, Kephart J, Hanson J, Kandasamy N, Jiang G (2009) Power and performance management of virtualized computing environments via lookahead control. Cluster Comput 12(1):1\u201315","journal-title":"Cluster Comput"},{"issue":"1","key":"5462_CR33","doi-asserted-by":"publisher","first-page":"54","DOI":"10.1145\/2541883.2541894","volume":"57","author":"J Lango","year":"2014","unstructured":"Lango J (2014) Toward software-defined SLAs. Commun ACM 57(1):54\u201360","journal-title":"Commun ACM"},{"key":"5462_CR34","doi-asserted-by":"publisher","first-page":"789","DOI":"10.1016\/j.future.2019.08.004","volume":"102","author":"Z Li","year":"2020","unstructured":"Li Z, Yu X, Yu L, Guo S, Chang V (2020) Energy-efficient and quality-aware VM consolidation method. Future Gener Comput Syst 102:789\u2013809","journal-title":"Future Gener Comput Syst"},{"key":"5462_CR35","doi-asserted-by":"crossref","unstructured":"Liu N, Li Z, Xu J, Xu Z, Lin S, Qiu Q, Tang J, Wang Y (2017) A hierarchical framework of cloud resource allocation and power management using deep reinforcement learning. In: International conference on distributed computing systems, pp 372\u2013382","DOI":"10.1109\/ICDCS.2017.123"},{"key":"5462_CR36","doi-asserted-by":"crossref","unstructured":"Machida F, Xiang J, Tadano K, Maeno Y (2012) Combined server rejuvenation in a virtualized data center. In: International conference on ubiquitous intelligence and computing and international conference on autonomic and trusted computing, pp 486\u2013493","DOI":"10.1109\/UIC-ATC.2012.52"},{"key":"5462_CR37","doi-asserted-by":"crossref","unstructured":"Malekloo M, Kara N (2014) Multi-objective ACO virtual machine placement in cloud computing environments. In: IEEE Globecom workshops, pp 112\u2013116","DOI":"10.1109\/GLOCOMW.2014.7063415"},{"key":"5462_CR38","unstructured":"Ma F, Liu F, Liu Z (2012) Multi-objective optimization for initial virtual machine placement in cloud data center. J Inform Comput Sci 9(16)"},{"key":"5462_CR39","doi-asserted-by":"publisher","first-page":"106","DOI":"10.1016\/j.jnca.2016.01.011","volume":"66","author":"M Masdari","year":"2016","unstructured":"Masdari M, Nabavi SS, Ahmadi V (2016) An overview of virtual machine placement schemes in cloud computing. J Netw Comput Appl 66:106\u2013127","journal-title":"J Netw Comput Appl"},{"key":"5462_CR40","doi-asserted-by":"crossref","unstructured":"Masdari M, Gharehpasha S, Ghobaei-Arani M, Ghasemi V (2019) Bio-inspired virtual machine placement schemes in cloud computing environment: taxonomy, review, and future research directions. Cluster Comput, 1\u201331","DOI":"10.1007\/s10586-019-03026-9"},{"key":"5462_CR41","doi-asserted-by":"crossref","unstructured":"Mills K, Filliben J, Dabrowski C (2011) Comparing VM-placement algorithms for on-demand clouds. In: Proceedings of international conference on cloud computing technology and Sci, pp 91\u201398","DOI":"10.1109\/CloudCom.2011.22"},{"key":"5462_CR42","unstructured":"Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D, Riedmiller MA (2013) Playing Atari with deep reinforcement learning. arXiv:1312.5602"},{"key":"5462_CR43","unstructured":"Panigrahy R, Talwar K, Uyeda L, Wieder U (2011) Heuristics for vector bin packing. Microsoft Research. http:\/\/research.microsoft.com\/apps\/pubs\/default.aspx?id=147927"},{"key":"5462_CR44","doi-asserted-by":"crossref","unstructured":"Papadopoulos AV, Maggio M (2015) Virtual machine migration in cloud infrastructures: problem formalization and policies proposal. In: Proceedings of conference on decision and control, pp 6698\u20136705","DOI":"10.1109\/CDC.2015.7403274"},{"key":"5462_CR45","unstructured":"Pashenkova E, Rish I, Dechter R (1996) Value iteration and policy iteration algorithms for Markov decision problem. In: AAAI\u201996: workshop on structural issues in planning and temporal reasoning, Citeseer"},{"issue":"2","key":"5462_CR46","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/2431211.2431216","volume":"45","author":"M Pearce","year":"2013","unstructured":"Pearce M, Zeadally S, Hunt R (2013) Virtualization: issues, security threats, and solutions. ACM Comput Surv 45(2):1\u201339","journal-title":"ACM Comput Surv"},{"key":"5462_CR47","doi-asserted-by":"publisher","first-page":"2370","DOI":"10.1007\/s10489-020-01633-3","volume":"50","author":"Y Qin","year":"2020","unstructured":"Qin Y, Wang H, Yi S, Li X, Zhai L (2020) Virtual machine placement based on multi-objective reinforcement learning. Appl Intell 50:2370\u20132383","journal-title":"Appl Intell"},{"key":"5462_CR48","unstructured":"Schaul T, Quan J, Antonoglou I, Silver D (2016) Prioritized experience replay. arXiv:1511.05952"},{"key":"5462_CR49","doi-asserted-by":"crossref","unstructured":"Shaw R, Howley E, Barrett E (2017) An advanced reinforcement learning approach for energy-aware virtual machine consolidation in cloud data centers. In: International conference for internet technology and secured transaction, pp 61\u201366","DOI":"10.23919\/ICITST.2017.8356347"},{"issue":"11","key":"5462_CR50","doi-asserted-by":"publisher","first-page":"2647","DOI":"10.1109\/TC.2013.148","volume":"63","author":"W Song","year":"2013","unstructured":"Song W, Xiao Z, Chen Q, Luo H (2013) Adaptive resource provisioning for the cloud using online bin packing. IEEE Trans Comput 63(11):2647\u20132660","journal-title":"IEEE Trans Comput"},{"key":"5462_CR51","unstructured":"Srikantaiah S, Kansal A, Zhao F (2008) Energy aware consolidation for cloud computing. In: USENIX HotPower08: Workshop on Power Aware Computing and Systems at OSDI, pp 1\u20135"},{"key":"5462_CR52","unstructured":"Sugerman J, Venkitachalam G, Lim B (2001) Virtualizing I\/O devices on VMware workstation\u2019s hosted virtual machine monitor. In: Proceedings of USENIX annual technical conference, pp 1\u201314"},{"issue":"2","key":"5462_CR53","doi-asserted-by":"publisher","first-page":"1960","DOI":"10.1109\/JIOT.2018.2871020","volume":"6","author":"Y Sun","year":"2018","unstructured":"Sun Y, Peng M, Mao S (2018) Deep reinforcement learning-based mode selection and resource management for green fog radio access networks. IEEE Internet Things J 6(2):1960\u20131971","journal-title":"IEEE Internet Things J"},{"key":"5462_CR54","unstructured":"Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. MIT Press, London"},{"issue":"1","key":"5462_CR55","first-page":"1","volume":"4","author":"C Szepesv\u00e1ri","year":"2010","unstructured":"Szepesv\u00e1ri C (2010) Algorithms for reinforcement learning. Synth Lect Artif Intell Mach Learn 4(1):1\u2013103","journal-title":"Synth Lect Artif Intell Mach Learn"},{"issue":"1","key":"5462_CR56","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1109\/JSYST.2013.2256731","volume":"8","author":"CW Tsai","year":"2013","unstructured":"Tsai CW, Rodrigues J (2013) Metaheuristic scheduling for cloud: a survey. IEEE Syst J 8(1):279\u2013291","journal-title":"IEEE Syst J"},{"key":"5462_CR57","doi-asserted-by":"publisher","first-page":"491","DOI":"10.1016\/j.procs.2016.02.093","volume":"78","author":"Z Usmani","year":"2016","unstructured":"Usmani Z, Singh S (2016) A survey of virtual machine placement techniques in a cloud data center. Proc Comput Sci 78:491\u2013498","journal-title":"Proc Comput Sci"},{"key":"5462_CR58","unstructured":"van Hasselt H (2010) Double Q-learning. In: Advances in neural information processing systems, pp 2613\u20132621"},{"key":"5462_CR59","doi-asserted-by":"crossref","unstructured":"van Hasselt H, Guez A, Silver D (2016) Deep reinforcement learning with double Q-learning. In: 30th AAAI conference on artificial intelligence","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"5462_CR60","unstructured":"van Hasselt H, Hessel M, Aslanides J (2019) When to use parametric models in reinforcement learning? arXiv:1906.05243"},{"key":"5462_CR61","doi-asserted-by":"publisher","first-page":"39974","DOI":"10.1109\/ACCESS.2019.2902846","volume":"7","author":"Y Wang","year":"2019","unstructured":"Wang Y, Liu H, Zheng W, Xia Y, Li Y, Chen P, Guo K, Xie H (2019) Multi-objective workflow scheduling with deep-Q-network-based multi-agent reinforcement learning. IEEE Access 7:39974\u201339982","journal-title":"IEEE Access"},{"key":"5462_CR62","unstructured":"Wang Z, Schaul T, Hessel M, van Hasselt H, Lanctot M, de\u00a0Freitas N (2016) Dueling network architectures for deep reinforcement learning. arXiv:1511.06581"},{"issue":"2","key":"5462_CR63","doi-asserted-by":"publisher","first-page":"95","DOI":"10.1016\/j.jpdc.2011.10.003","volume":"72","author":"CZ Xu","year":"2012","unstructured":"Xu CZ, Rao J, Bu X (2012) Url: a unified reinforcement learning approach for autonomic cloud management. J Parallel Distrib Comput 72(2):95\u2013105","journal-title":"J Parallel Distrib Comput"},{"issue":"12","key":"5462_CR64","doi-asserted-by":"publisher","first-page":"e4123","DOI":"10.1002\/cpe.4123","volume":"29","author":"M Xu","year":"2017","unstructured":"Xu M, Tian W, Buyya R (2017) A survey on load balancing algorithms for virtual machines placement in cloud computing. Concurr Comput Pract Exp 29(12):e4123","journal-title":"Concurr Comput Pract Exp"},{"key":"5462_CR65","doi-asserted-by":"publisher","unstructured":"Yuan X, Sun M, Lou W (2020) A dynamic deep-learning-based virtual edge node placement scheme for edge cloud systems in mobile environment. IEEE Trans Cloud Comput. https:\/\/doi.org\/10.1109\/TCC.2020.2974948","DOI":"10.1109\/TCC.2020.2974948"},{"issue":"3","key":"5462_CR66","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1002\/ett.2640","volume":"24","author":"X Zhang","year":"2013","unstructured":"Zhang X, Zhang J, Huang Y, Wang W (2013) On the study of fundamental trade-offs between QoE and energy efficiency in wireless networks. Trans Emerg Telecommun Technol 24(3):259\u2013265","journal-title":"Trans Emerg Telecommun Technol"},{"issue":"2","key":"5462_CR67","doi-asserted-by":"publisher","first-page":"1206","DOI":"10.1109\/COMST.2018.2794881","volume":"20","author":"F Zhang","year":"2018","unstructured":"Zhang F, Liu G, Fu X, Yahyapour R (2018) A survey on virtual machine migration: challenges, techniques, and open issues. IEEE Commun Surv Tutor 20(2):1206\u20131243","journal-title":"IEEE Commun Surv Tutor"},{"key":"5462_CR68","doi-asserted-by":"publisher","first-page":"3319","DOI":"10.1109\/ACCESS.2019.2963051","volume":"8","author":"J Zhao","year":"2020","unstructured":"Zhao J, Kong M, Li Q, Sun X (2020) Contract-based computing resource management via deep reinforcement learning in vehicular fog computing. IEEE Access 8:3319\u20133329","journal-title":"IEEE Access"}],"updated-by":[{"DOI":"10.1007\/s00500-020-05536-w","type":"correction","label":"Correction","source":"publisher","updated":{"date-parts":[[2021,1,15]],"date-time":"2021-01-15T00:00:00Z","timestamp":1610668800000}}],"container-title":["Soft Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00500-020-05462-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00500-020-05462-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00500-020-05462-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,4]],"date-time":"2022-12-04T10:29:55Z","timestamp":1670149795000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00500-020-05462-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,12]]},"references-count":68,"journal-issue":{"issue":"19","published-print":{"date-parts":[[2021,10]]}},"alternative-id":["5462"],"URL":"https:\/\/doi.org\/10.1007\/s00500-020-05462-x","relation":{"correction":[{"id-type":"doi","id":"10.1007\/s00500-020-05536-w","asserted-by":"object"}]},"ISSN":["1432-7643","1433-7479"],"issn-type":[{"value":"1432-7643","type":"print"},{"value":"1433-7479","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12,12]]},"assertion":[{"value":"12 December 2020","order":1,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 January 2021","order":2,"name":"change_date","label":"Change Date","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Correction","order":3,"name":"change_type","label":"Change Type","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"A Correction to this paper has been published:","order":4,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"https:\/\/doi.org\/10.1007\/s00500-020-05536-w","URL":"https:\/\/doi.org\/10.1007\/s00500-020-05536-w","order":5,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Compliance with ethical standards"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"This article does not contain any studies with human participants or animals performed by any of the authors.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}}]}}