{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,8]],"date-time":"2026-07-08T07:27:15Z","timestamp":1783495635926,"version":"3.55.0"},"reference-count":42,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2021,6,21]],"date-time":"2021-06-21T00:00:00Z","timestamp":1624233600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Media Development Authority"},{"DOI":"10.13039\/501100001321","name":"National Research Foundation","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001321","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Green Data Centre Programme"},{"name":"Energy Programme"},{"name":"Energy Market Authority","award":["NRF2017EWT-EP003-023"],"award-info":[{"award-number":["NRF2017EWT-EP003-023"]}]},{"name":"Green Data Centre Research"},{"name":"Behavioural Studies in the Energy, Water, Waste and Transportation Sectors","award":["BSEWWT2017_2_06"],"award-info":[{"award-number":["BSEWWT2017_2_06"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Sen. Netw."],"published-print":{"date-parts":[[2021,8,31]]},"abstract":"<jats:p>Air free-cooled data centers (DCs) have not existed in the tropical zone due to the unique challenges of year-round high ambient temperature and relative humidity (RH). The increasing availability of servers that can tolerate higher temperatures and RH due to the regulatory bodies\u2019 prompts to raise DC temperature setpoints sheds light upon the feasibility of air free-cooled DCs in the tropics. However, due to the complex psychrometric dynamics, operating the air free-cooled DC in the tropics generally requires adaptive control of supply air condition to maintain the computing performance and reliability of the servers. This article studies the problem of controlling the supply air temperature and RH in a free-cooled tropical DC below certain thresholds. To achieve the goal, we formulate the control problem as Markov decision processes and apply deep reinforcement learning (DRL) to learn the control policy that minimizes the cooling energy while satisfying the requirements on the supply air temperature and RH. We also develop a constrained DRL solution for performance improvements. Extensive evaluation based on real data traces collected from an air free-cooled testbed and comparisons among the unconstrained and constrained DRL approaches as well as two other baseline approaches show the superior performance of our proposed solutions.<\/jats:p>","DOI":"10.1145\/3439332","type":"journal-article","created":{"date-parts":[[2021,6,21]],"date-time":"2021-06-21T20:19:31Z","timestamp":1624306771000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":25,"title":["Deep Reinforcement Learning for Tropical Air Free-cooled Data Center Control"],"prefix":"10.1145","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0115-8726","authenticated-orcid":false,"given":"Duc Van","family":"Le","sequence":"first","affiliation":[{"name":"Computer Science and Engineering, Nanyang Technological University, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rongrong","family":"Wang","sequence":"additional","affiliation":[{"name":"Computer Science and Engineering, Nanyang Technological University, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yingbo","family":"Liu","sequence":"additional","affiliation":[{"name":"Computer Science and Engineering, Nanyang Technological University, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8441-9973","authenticated-orcid":false,"given":"Rui","family":"Tan","sequence":"additional","affiliation":[{"name":"Computer Science and Engineering, Nanyang Technological University, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yew-Wah","family":"Wong","sequence":"additional","affiliation":[{"name":"Energy Research Institute, Nanyang Technological University, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yonggang","family":"Wen","sequence":"additional","affiliation":[{"name":"Computer Science and Engineering, Nanyang Technological University, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,6,21]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Uptime Institute. 2014. Data Center Industry Survey Results. Retrieved from https:\/\/journal.uptimeinstitute.com\/2014-data-centerindustry-survey\/.  Uptime Institute. 2014. Data Center Industry Survey Results. Retrieved from https:\/\/journal.uptimeinstitute.com\/2014-data-centerindustry-survey\/."},{"key":"e_1_2_1_2_1","volume-title":"Backblaze hard drive stats for","year":"2018","unstructured":"Backblaze. 2018. Backblaze hard drive stats for 2018 . Retrieved from https:\/\/www.backblaze.com\/blog\/hard-drive-stats-for-2018\/. Backblaze. 2018. Backblaze hard drive stats for 2018. Retrieved from https:\/\/www.backblaze.com\/blog\/hard-drive-stats-for-2018\/."},{"key":"e_1_2_1_3_1","volume-title":"Thermal guidelines for data processing environments\u2014Expanded data center classes and usage guidance. White Paper","author":"ASHRAE TC","year":"2011","unstructured":"ASHRAE TC 9.9. 2011. Thermal guidelines for data processing environments\u2014Expanded data center classes and usage guidance. White Paper ( 2011 ). Retrieved from https:\/\/airatwork.com\/wp-content\/uploads\/ASHRAETC99.pdf. ASHRAE TC 9.9. 2011. Thermal guidelines for data processing environments\u2014Expanded data center classes and usage guidance. White Paper (2011). Retrieved from https:\/\/airatwork.com\/wp-content\/uploads\/ASHRAETC99.pdf."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.5555\/3305381.3305384"},{"key":"e_1_2_1_5_1","volume-title":"Constrained Markov Decision Processes","author":"Altman Aitan","unstructured":"Aitan Altman . 1999. Constrained Markov Decision Processes . Chapman & Hall\/CRC. Aitan Altman. 1999. Constrained Markov Decision Processes. Chapman & Hall\/CRC."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.5555\/3305381.3305396"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.5555\/3327757.3327922"},{"key":"e_1_2_1_8_1","volume-title":"Gaseous and particulate contamination guidelines for data centers. White Paper","author":"ASHRAE.","year":"2011","unstructured":"ASHRAE. 2011. Gaseous and particulate contamination guidelines for data centers. White Paper ( 2011 ). ASHRAE. 2011. Gaseous and particulate contamination guidelines for data centers. White Paper (2011)."},{"key":"e_1_2_1_9_1","unstructured":"ASHRAE. 2013. Psychrometrics. In 2013 ASHRAE Handbook: Fundamentals (SI).  ASHRAE. 2013. Psychrometrics. In 2013 ASHRAE Handbook: Fundamentals (SI)."},{"key":"e_1_2_1_10_1","volume-title":"Nonlinear Programming","author":"Bertsekas Dimitri P.","unstructured":"Dimitri P. Bertsekas . 1999. Nonlinear Programming . Athena Scientific . Dimitri P. Bertsekas. 1999. Nonlinear Programming. Athena Scientific."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3360322.3360849"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.enbuild.2013.09.013"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.5555\/3122009.3242024"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/CloudCom.2011.100"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2499368.2451123"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2786763.2694378"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3307772.3328307"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.rser.2014.04.017"},{"key":"e_1_2_1_19_1","unstructured":"JEDEC. 2019. Arrhenius equation (for reliability). Retrieved from https:\/\/www.jedec.org\/standards-documents\/dictionary\/terms\/arrh% enius-equation-reliability.  JEDEC. 2019. Arrhenius equation (for reliability). Retrieved from https:\/\/www.jedec.org\/standards-documents\/dictionary\/terms\/arrh% enius-equation-reliability."},{"key":"e_1_2_1_20_1","unstructured":"Will Knight. 2018. Google just gave control over data center cooling to an AI. Retrieved from https:\/\/www.technologyreview.com\/2018\/08\/17\/1409% 87\/google-just-gave-control-over-data-center-cooling-to-an-ai\/.  Will Knight. 2018. Google just gave control over data center cooling to an AI. Retrieved from https:\/\/www.technologyreview.com\/2018\/08\/17\/1409% 87\/google-just-gave-control-over-data-center-cooling-to-an-ai\/."},{"key":"e_1_2_1_21_1","unstructured":"Keith James Laidler and J. Keith. 1965. Chemical Kinetics. Vol. 15. McGraw-Hill.  Keith James Laidler and J. Keith. 1965. Chemical Kinetics. Vol. 15. McGraw-Hill."},{"key":"e_1_2_1_22_1","volume-title":"Tropical Data Centre Proof-of-Concept. Techincal Report","author":"Le Duc Van","unstructured":"Duc Van Le , Yingbo Liu , Rongrong Wang , and Rui Tan . 2019. Tropical Data Centre Proof-of-Concept. Techincal Report . Nanyang Technological University . Retrieved from https:\/\/dr.ntu.edu.sg\/\/handle\/10356\/137780. Duc Van Le, Yingbo Liu, Rongrong Wang, and Rui Tan. 2019. Tropical Data Centre Proof-of-Concept. Techincal Report. Nanyang Technological University. Retrieved from https:\/\/dr.ntu.edu.sg\/\/handle\/10356\/137780."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.enbuild.2013.04.013"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2019.2927410"},{"key":"e_1_2_1_25_1","unstructured":"Qingkai Liang Fanyu Que and Eytan Modiano. 2018. Accelerated Primal-Dual Policy Optimization for Safe Reinforcement Learning. Retrieved from https:\/\/arxiv.org\/abs\/1802.06480.  Qingkai Liang Fanyu Que and Eytan Modiano. 2018. Accelerated Primal-Dual Policy Optimization for Safe Reinforcement Learning. Retrieved from https:\/\/arxiv.org\/abs\/1802.06480."},{"key":"e_1_2_1_26_1","unstructured":"Timothy P. Lillicrap Jonathan J. Hunt Alexander Pritzel Nicolas Heess Tom Erez Yuval Tassa David Silver and Daan Wierstra. 2015. Continuous control with deep reinforcement learning. Retrieved from https:\/\/arxiv.org\/abs\/1509.02971.  Timothy P. Lillicrap Jonathan J. Hunt Alexander Pritzel Nicolas Heess Tom Erez Yuval Tassa David Silver and Daan Wierstra. 2015. Continuous control with deep reinforcement learning. Retrieved from https:\/\/arxiv.org\/abs\/1509.02971."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2806777.2806938"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.5555\/2930583.2930588"},{"key":"e_1_2_1_29_1","volume-title":"Human level control through deep reinforcement learning. Nature 518, 7540","author":"\u00a0al V Mnih","year":"2015","unstructured":"V Mnih et \u00a0al . 2015. Human level control through deep reinforcement learning. Nature 518, 7540 ( 2015 ), 529\u2013533. V Mnih et\u00a0al. 2015. Human level control through deep reinforcement learning. Nature 518, 7540 (2015), 529\u2013533."},{"key":"e_1_2_1_30_1","volume-title":"Economizer modes of data center cooling systems. Schneider 886 Electric White Paper Library","author":"Niemann John","year":"2012","unstructured":"John Niemann , John Bean , and Victor Avelar . 2012. Economizer modes of data center cooling systems. Schneider 886 Electric White Paper Library ( 2012 ). Retrieved from https:\/\/download.schneider-electric.com\/files?p_Doc_Ref=SPD_JNIN-7RMNRX_EN. John Niemann, John Bean, and Victor Avelar. 2012. Economizer modes of data center cooling systems. Schneider 886 Electric White Paper Library (2012). Retrieved from https:\/\/download.schneider-electric.com\/files?p_Doc_Ref=SPD_JNIN-7RMNRX_EN."},{"key":"e_1_2_1_31_1","unstructured":"Jay Park. 2011. Designing a Very Efficient Data Center. Retrieved from https:\/\/engineering.fb.com\/2011\/04\/14\/core-data\/designing-a-very-efficient-data-center\/.  Jay Park. 2011. Designing a Very Efficient Data Center. Retrieved from https:\/\/engineering.fb.com\/2011\/04\/14\/core-data\/designing-a-very-efficient-data-center\/."},{"key":"e_1_2_1_32_1","unstructured":"Matthias Plappert. 2016. keras-rl. Retrieved from https:\/\/github.com\/keras-rl\/keras-rl.  Matthias Plappert. 2016. keras-rl. Retrieved from https:\/\/github.com\/keras-rl\/keras-rl."},{"key":"e_1_2_1_33_1","unstructured":"John Rath. 2011. Data Centers Can Be Even Warmer. Retrieved from https:\/\/www.datacenterknowledge.com\/archives\/201% 1\/02\/22\/ashrae-data-centers-can-be-even-warmer.  John Rath. 2011. Data Centers Can Be Even Warmer. Retrieved from https:\/\/www.datacenterknowledge.com\/archives\/201% 1\/02\/22\/ashrae-data-centers-can-be-even-warmer."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1115\/IPACK2015-48176"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.5555\/551283"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1149\/1.2220797"},{"key":"e_1_2_1_37_1","unstructured":"Chen Tessler Daniel J. Mankowitz and Shie Mannor. 2018. Reward Constrained Policy Optimization. Retrieved from https:\/\/arxiv.org\/abs\/1805.11074.  Chen Tessler Daniel J. Mankowitz and Shie Mannor. 2018. Reward Constrained Policy Optimization. Retrieved from https:\/\/arxiv.org\/abs\/1805.11074."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3360322.3360845"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3061639.3062224"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDCS.2019.00069"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/COMST.2016.2545109"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3276774.3276775"}],"container-title":["ACM Transactions on Sensor Networks"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3439332","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3439332","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:03:08Z","timestamp":1750197788000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3439332"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,21]]},"references-count":42,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2021,8,31]]}},"alternative-id":["10.1145\/3439332"],"URL":"https:\/\/doi.org\/10.1145\/3439332","relation":{},"ISSN":["1550-4859","1550-4867"],"issn-type":[{"value":"1550-4859","type":"print"},{"value":"1550-4867","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,21]]},"assertion":[{"value":"2020-07-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-11-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-06-21","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}