{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T02:54:36Z","timestamp":1760151276051,"version":"build-2065373602"},"reference-count":40,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2022,3,2]],"date-time":"2022-03-02T00:00:00Z","timestamp":1646179200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["71932002;  61573206; 11931018"],"award-info":[{"award-number":["71932002;  61573206; 11931018"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Beijing Social Science Foundation Research Base Project","award":["19JDGLA004"],"award-info":[{"award-number":["19JDGLA004"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Systems"],"abstract":"<jats:p>In this paper, we apply a Markov decision process to find the optimal asynchronous dynamic policy of an energy-efficient data center with two server groups. Servers in Group 1 always work, while servers in Group 2 may either work or sleep, and a fast setup process occurs when the server\u2019s states are changed from sleep to work. The servers in Group 1 are faster and cheaper than those of Group 2 so that Group 1 has a higher service priority. Putting each server in Group 2 to sleep can reduce system costs and energy consumption, but it must bear setup costs and transfer costs. For such a data center, an asynchronous dynamic policy is designed as two sub-policies: The setup policy and the sleep policy, both of which determine the switch rule between the work and sleep states for each server in Group 2. To find the optimal asynchronous dynamic policy, we apply the sensitivity-based optimization to establish a block-structured policy-based Markov process and use a block-structured policy-based Poisson equation to compute the unique solution of the performance potential by means of the RG-factorization. Based on this, we can characterize the monotonicity and optimality of the long-run average profit of the data center with respect to the asynchronous dynamic policy under different service prices. Furthermore, we prove that a bang\u2013bang control is always optimal for this optimization problem. We hope that the methodology and results developed in this paper can shed light on the study of more general energy-efficient data centers.<\/jats:p>","DOI":"10.3390\/systems10020027","type":"journal-article","created":{"date-parts":[[2022,3,2]],"date-time":"2022-03-02T22:53:56Z","timestamp":1646261636000},"page":"27","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Optimal Asynchronous Dynamic Policies in Energy-Efficient Data Centers"],"prefix":"10.3390","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0396-1232","authenticated-orcid":false,"given":"Jing-Yu","family":"Ma","sequence":"first","affiliation":[{"name":"Business School, Xuzhou University of Technology, Xuzhou 221018, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Quan-Lin","family":"Li","sequence":"additional","affiliation":[{"name":"School of Economics and Management, Beijing University of Technology, Beijing 100124, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Li","family":"Xia","sequence":"additional","affiliation":[{"name":"School of Business, Sun Yat-sen University, Guangzhou 510275, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,3,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"984","DOI":"10.1126\/science.aba3758","article-title":"Recalibrating global data center energy-use estimates","volume":"367","author":"Masanet","year":"2020","journal-title":"Science"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"102253","DOI":"10.1016\/j.sysarc.2021.102253","article-title":"A survey on data center cooling systems: Technology, power consumption modeling and control strategy optimization","volume":"119","author":"Zhang","year":"2021","journal-title":"J. Syst. Archit."},{"key":"ref_3","first-page":"14","article-title":"A review of thermal management and innovative cooling strategies for data center","volume":"19","author":"Nadjahi","year":"2018","journal-title":"Sustain. Comput. Inform. Syst."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"116798","DOI":"10.1016\/j.apenergy.2021.116798","article-title":"Usage impact on data center electricity needs: A system dynamic forecasting model","volume":"291","author":"Koot","year":"2021","journal-title":"Appl. Energy"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"7545","DOI":"10.1007\/s11227-020-03180-7","article-title":"Performance issues and solutions in SDN-based data center: A survey","volume":"76","author":"Shirmarz","year":"2020","journal-title":"J. Supercomput."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Li, Q.L., Ma, J.Y., Xie, M.Z., and Xia, L. (2017, January 21\u201323). Group-server queues. Proceedings of the International Conference on Queueing Theory and Network Applications, Qinhuangdao, China.","DOI":"10.1007\/978-3-319-68520-5_4"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/s11134-020-09684-6","article-title":"Open problems in queueing theory inspired by data center computing","volume":"97","year":"2021","journal-title":"Queueing Syst."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1109\/MC.2007.443","article-title":"The case for energy-proportional computing","volume":"40","author":"Barroso","year":"2007","journal-title":"Computer"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"497","DOI":"10.1016\/j.adhoc.2014.11.013","article-title":"Automatic energy efficiency management of data center resources by load-dependent server activation and sleep modes","volume":"25","author":"Kuehn","year":"2015","journal-title":"Ad Hoc Netw."},{"key":"ref_10","unstructured":"Gandhi, A. (2013). Dynamic Server Provisioning for Data Center Power Management. [Ph.D. Thesis, School of Computer Science, Carnegie Mellon University]."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1007\/s11134-014-9409-7","article-title":"Exact analysis of the M\/M\/k\/setup class of Markov chains via recursive renewal reward","volume":"77","author":"Gandhi","year":"2014","journal-title":"Queueing Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1155","DOI":"10.1016\/j.peva.2010.08.009","article-title":"Optimality analysis of energy-performance trade-off for server farm management","volume":"67","author":"Gandhi","year":"2010","journal-title":"Perform. Eval."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1123","DOI":"10.1016\/j.peva.2010.07.004","article-title":"Server farms with setup costs","volume":"67","author":"Gandhi","year":"2010","journal-title":"Perform. Eval."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1016\/j.peva.2015.04.002","article-title":"On optimal policies for energy-aware servers","volume":"90","author":"Maccio","year":"2015","journal-title":"Perform. Eval."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Maccio, V.J., and Down, D.G. (2016, January 19\u201321). Exact analysis of energy-aware multiserver queueing systems with setup times. Proceedings of the IEEE 24th International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems, London, UK.","DOI":"10.1109\/MASCOTS.2016.47"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1007\/s11235-016-0177-z","article-title":"Exact solutions for M\/M\/c\/setup queues","volume":"64","year":"2017","journal-title":"Telecommun. Syst."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Phung-Duc, T., and Kawanishi, K.I. (2016, January 24\u201326). Energy-aware data centers with s-staggered setup and abandonment. Proceedings of the International Conference on Analytical and Stochastic Modeling Techniques and Applications, Cardiff, UK.","DOI":"10.1007\/978-3-319-43904-4_19"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1016\/j.peva.2016.06.003","article-title":"Optimal energy-aware control policies for FIFO servers","volume":"103","author":"Gebrehiwot","year":"2016","journal-title":"Perform. Eval."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1016\/j.orl.2015.12.004","article-title":"Energy-performance trade-off for processor sharing queues with setup delay","volume":"44","author":"Gebrehiwot","year":"2016","journal-title":"Oper. Res. Lett."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1016\/j.peva.2017.07.003","article-title":"Energy-aware SRPT server with batch arrivals: Analysis and optimization","volume":"15","author":"Gebrehiwot","year":"2017","journal-title":"Perform. Eval."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1222","DOI":"10.1016\/j.peva.2011.07.017","article-title":"Service center trade-offs between customer impatience and power consumption","volume":"68","author":"Mitrani","year":"2011","journal-title":"Perform. Eval."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1007\/s10479-011-0932-1","article-title":"Managing performance and power consumption in a server farm","volume":"202","author":"Mitrani","year":"2013","journal-title":"Ann. Oper. Res."},{"key":"ref_23","first-page":"356","article-title":"Optimal sleeping: Models and experiments for energy-delay tradeoff","volume":"4","author":"Kamitsos","year":"2017","journal-title":"Int. J. Syst. Sci. Oper. Logist."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"585","DOI":"10.1287\/opre.36.4.585","article-title":"Decision processes with monotone hysteretic policies","volume":"36","author":"Hipp","year":"1988","journal-title":"Oper. Res."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1116","DOI":"10.1287\/opre.32.5.1116","article-title":"M\/M\/1 queueing decision processes with monotone hysteretic optimal policies","volume":"32","author":"Lu","year":"1984","journal-title":"Oper. Res."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1044","DOI":"10.1109\/TCST.2016.2575801","article-title":"Online learning-based server provisioning for electricity cost reduction in data center","volume":"25","author":"Yang","year":"2017","journal-title":"IEEE Trans. Control Syst. Technol."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1016\/j.future.2020.02.018","article-title":"Q-learning based dynamic task scheduling for energy-efficient cloud computing","volume":"108","author":"Ding","year":"2020","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"2077","DOI":"10.1111\/poms.13355","article-title":"Data center network design for internet-related services and cloud computing","volume":"30","author":"Liang","year":"2021","journal-title":"Prod. Oper. Manag."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"3290","DOI":"10.1109\/TAC.2018.2799576","article-title":"Dynamic pricing control for open queueing networks","volume":"63","author":"Xia","year":"2018","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"5111","DOI":"10.1109\/TAC.2017.2678109","article-title":"Service rate control of tandem queues with power constraints","volume":"62","author":"Xia","year":"2017","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"567","DOI":"10.1007\/s10626-019-00293-x","article-title":"Optimal energy-efficient policies for data centers through sensitivity-based optimization","volume":"29","author":"Ma","year":"2019","journal-title":"Discrete Event Dyn. Syst."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Chi, C., Ji, K., Marahatta, A., Song, P., Zhang, F., and Liu, Z. (2020, January 22\u201326). Jointly optimizing the IT and cooling systems for data center energy efficiency based on multi-agent deep reinforcement learning. Proceedings of the 11th ACM International Conference on Future Energy Systems, Virtual Event, Australia.","DOI":"10.1145\/3396851.3402658"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Li, Q.L. (2010). Constructive Computation in Stochastic Models with Applications: The RG-Factorizations, Springer.","DOI":"10.1007\/978-3-642-11492-2"},{"key":"ref_34","unstructured":"Cao, X.R. (2007). Stochastic Learning and Optimization\u2014A Sensitivity-Based Approach, Springer."},{"key":"ref_35","unstructured":"Xia, L., Zhang, Z.G., and Li, Q.L. (2021). A c\/\u03bc-rule for for job assignment in heterogeneous group-server queues. Prod. Oper. Manag., 1\u201318."},{"key":"ref_36","unstructured":"Li, Q.L., Li, Y.M., Ma, J.Y., and Liu, H.L. (2019). A complete algebraic transformational solution for the optimal dynamic policy in inventory rationing across two demand classes. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Ma, J.Y., and Li, Q.L. (2021, January 20\u201322). Sensitivity-based optimization for blockchain selfish mining. Proceedings of the International Conference on Algorithmic Aspects of Information and Management, Dallas, TX, USA.","DOI":"10.1007\/978-3-030-93176-6_28"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"2808","DOI":"10.1111\/poms.13252","article-title":"Risk-sensitive Markov decision processes with combined metrics of mean and variance","volume":"29","author":"Xia","year":"2020","journal-title":"Prod. Oper. Manag."},{"key":"ref_39","unstructured":"Puterman, M.L. (2014). Markov Decision Processes: Discrete Stochastic Dynamic Programming, John Wiley& Sons."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"204","DOI":"10.1214\/17-AAP1303","article-title":"Diffusion approximations for controlled weakly interacting large finite state systems with simultaneous jumps","volume":"28","author":"Budhiraja","year":"2018","journal-title":"Ann. Appl. Probab."}],"container-title":["Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-8954\/10\/2\/27\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:31:02Z","timestamp":1760135462000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-8954\/10\/2\/27"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,2]]},"references-count":40,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2022,4]]}},"alternative-id":["systems10020027"],"URL":"https:\/\/doi.org\/10.3390\/systems10020027","relation":{},"ISSN":["2079-8954"],"issn-type":[{"type":"electronic","value":"2079-8954"}],"subject":[],"published":{"date-parts":[[2022,3,2]]}}}