{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,10]],"date-time":"2026-08-10T10:28:44Z","timestamp":1786357724888,"version":"build-2736575974"},"reference-count":35,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2026,8,7]],"date-time":"2026-08-07T00:00:00Z","timestamp":1786060800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Guangzhou Power Supply Bureau"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The sustained growth of cloud computing and AI training workloads drives data center expansion. Optimizing their control is therefore critical for reducing operational costs. Edge real-time control is indispensable for guaranteeing thermal safety, data sovereignty, and offline availability. Yet deploying Deep Reinforcement Learning (DRL) in production Heating, Ventilation, and Air Conditioning (HVAC) environments confronts cold-start risks, edge\u2013cloud computational asymmetry, and multi-objective conflicts spanning energy efficiency, electricity cost, and thermal safety. To address these challenges, this paper proposes an edge-cloud collaborative physics-informed reinforcement learning framework for production data center HVAC control. The framework integrates a physics-informed cold-start solution using Adaptive Particle Swarm Optimization (APSO) to generate physically constrained initial policies on a gray-box digital twin without expert demonstration data, a three-time-scale edge\u2013cloud architecture coordinating minute-level edge Soft Actor-Critic (SAC) real-time inference, weekly edge APSO online model identification, daily cloud Non-dominated Sorting Genetic Algorithm III (NSGA-III) thermal storage scheduling, and a constraint-aware safe projection layer that embeds thermal safety hard constraints directly into the neural network policy. The framework is validated through a seven-month production deployment spanning the complete summer-to-winter transition, comprising approximately 3.2 million sensor records and evaluated with rigorous statistical methods.<\/jats:p>","DOI":"10.3390\/s26165031","type":"journal-article","created":{"date-parts":[[2026,8,7]],"date-time":"2026-08-07T16:00:43Z","timestamp":1786118443000},"page":"5031","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Multi-Objective Optimization for Data Center HVAC Systems Based on Edge\u2013Cloud Collaborative Deep Reinforcement Learning"],"prefix":"10.3390","volume":"26","author":[{"given":"Shichao","family":"Huang","sequence":"first","affiliation":[{"name":"School of Electronic and Information Engineering, South China University of Technology, Guangzhou 510000, China"},{"name":"Guangzhou Power Supply Bureau, Guangzhou 510000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yibing","family":"Zhou","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, South China University of Technology, Guangzhou 510000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuan","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Electronic and Information Engineering, South China University of Technology, Guangzhou 510000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,8,7]]},"reference":[{"key":"ref_1","unstructured":"International Energy Agency (2024). Electricity 2024: Analysis and Forecast to 2026, International Energy Agency. Technical report."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"637","DOI":"10.1109\/JIOT.2016.2579198","article-title":"Edge Computing: Vision and Challenges","volume":"3","author":"Shi","year":"2016","journal-title":"IEEE Internet Things J."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1109\/JIOT.2017.2750180","article-title":"Mobile Edge Computing: A Survey","volume":"5","author":"Abbas","year":"2018","journal-title":"IEEE Internet Things J."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"6900","DOI":"10.1109\/ACCESS.2017.2778504","article-title":"A Survey on the Edge Computing for the Internet of Things","volume":"6","author":"Yu","year":"2018","journal-title":"IEEE Access"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1072","DOI":"10.1016\/j.apenergy.2018.11.002","article-title":"Reinforcement learning for demand response: A review of algorithms and modeling techniques","volume":"235","author":"Nagy","year":"2019","journal-title":"Appl. Energy"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Sutton, R.S., and Barto, A.G. (1998). Reinforcement Learning: An Introduction, MIT Press.","DOI":"10.1109\/TNN.1998.712192"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-Level Control Through Deep Reinforcement Learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1016\/j.arcontrol.2020.09.001","article-title":"All you need to know about model predictive control for buildings","volume":"50","author":"Arroyo","year":"2020","journal-title":"Annu. Rev. Control"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Hedayat, S., Ziarati, T., and Manganelli, M. (2025). A Physics-Informed Reinforcement Learning Framework for HVAC Optimization: Thermodynamically-Constrained Deep Deterministic Policy Gradients with Simulation-Based Validation. Energies, 18.","DOI":"10.20944\/preprints202511.0139.v1"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Esmaeili, M., Hammes, S., Tosatto, S., Geisler-Moroder, D., Gao, Y., Pritoni, M., Li, B., and Hug, G. (2025). Safe Reinforcement Learning for Buildings: Minimizing Energy Use While Maximizing Occupant Comfort. Energies, 18.","DOI":"10.3390\/en18195313"},{"key":"ref_11","first-page":"118491","article-title":"Safe deep reinforcement learning for building energy management","volume":"377","author":"Wang","year":"2025","journal-title":"Appl. Energy"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., and Kalenichenko, D. (2018, January 18\u201323). Quantization and training of neural networks for efficient integer-arithmetic-only inference. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00286"},{"key":"ref_13","unstructured":"Cai, H., Gan, C., Wang, T., Zhang, Z., and Han, S. (2019). Once-for-all: Train one network and specialize it for efficient deployment. arXiv."},{"key":"ref_14","unstructured":"Levine, S., Kumar, A., Tucker, G., and Fu, J. (2020). Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems. arXiv."},{"key":"ref_15","unstructured":"Zhan, X., Zhu, X., Cheng, P., Hu, X., He, Z., Geng, H., Leng, J., Zheng, H., Liu, C., and Hong, T. (2025, January 24\u201328). Data Center Cooling System Optimization Using Offline Reinforcement Learning. Proceedings of the International Conference on Learning Representations (ICLR), Singapore."},{"key":"ref_16","first-page":"1","article-title":"Green data center cooling control via physics-guided safe reinforcement learning","volume":"8","author":"Wang","year":"2024","journal-title":"ACM Trans.-Cyber-Phys. Syst."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Cao, Z., Wang, R., Zhou, X., and Wen, Y. (2023). Toward model-assisted safe reinforcement learning for data center cooling control: A Lyapunov-based approach. Proceedings of the 14th ACM International Conference on Future Energy Systems (e-Energy), ACM.","DOI":"10.1145\/3575813.3597343"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1109\/TPWRS.2023.3237888","article-title":"District cooling system control for providing operating reserve based on safe deep reinforcement learning","volume":"39","author":"Yu","year":"2023","journal-title":"IEEE Trans. Power Syst."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"132202","DOI":"10.1007\/s11432-023-3872-9","article-title":"Constrained reinforcement learning with statewise projection: A control barrier function approach","volume":"67","author":"Jin","year":"2024","journal-title":"Sci. China Inf. Sci."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"118118","DOI":"10.1016\/j.enconman.2024.118118","article-title":"A hierarchical HVAC optimal control method for reducing energy consumption and improving indoor air quality incorporating soft Actor-Critic and hybrid search optimization","volume":"302","author":"Cui","year":"2024","journal-title":"Energy Convers. Manag."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"472","DOI":"10.1016\/j.enbuild.2019.07.029","article-title":"Whole building energy model for HVAC optimal control: A practical framework based on deep reinforcement learning","volume":"199","author":"Zhang","year":"2019","journal-title":"Energy Build."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"22150","DOI":"10.1609\/aaai.v38i20.30219","article-title":"Active reinforcement learning for robust building control","volume":"Volume 38","author":"Jang","year":"2024","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"V\u00e1zquez-Canteli, J.R., K\u00e4mpf, J., Henze, G., and Nagy, Z. (2019, January 13\u201314). Citylearn v1. 0: An openai gym environment for demand response with deep reinforcement learning. Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, New York, NY, USA.","DOI":"10.1145\/3360322.3360998"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1427","DOI":"10.1080\/23744731.2020.1795514","article-title":"The ASHRAE Great Energy Predictor III competition: Overview and results","volume":"26","author":"Miller","year":"2020","journal-title":"Sci. Technol. Built Environ."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1655","DOI":"10.1109\/JPROC.2019.2921977","article-title":"Deep learning with edge computing: A review","volume":"107","author":"Chen","year":"2019","journal-title":"Proc. IEEE"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"11858","DOI":"10.1109\/JIOT.2021.3132033","article-title":"Distributed Optimal Control for HVAC Systems Adopting Edge Computing-Strategy, Implementation, and Experimental Validation","volume":"9","author":"Su","year":"2022","journal-title":"IEEE Internet Things J."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Chen, B., Cai, Z., and Berg\u00e9s, M. (2019, January 13\u201314). Gnu-rl: A precocial reinforcement learning solution for building hvac control using a differentiable mpc policy. Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, New York, NY, USA.","DOI":"10.1145\/3360322.3360849"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"123969","DOI":"10.1016\/j.apenergy.2024.123969","article-title":"Economic model predictive control for building HVAC system: A comparative analysis of model-based and data-driven approaches using the BOPTEST Framework","volume":"374","author":"Zheng","year":"2024","journal-title":"Appl. Energy"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"138389","DOI":"10.1016\/j.energy.2025.138389","article-title":"Optimization of a free cooling system integrated with cold thermal energy storage in data center based on model predictive control","volume":"336","author":"Xiang","year":"2025","journal-title":"Energy"},{"key":"ref_30","unstructured":"ASHRAE (2021). ANSI\/ASHRAE Guideline 36-2021: High-Performance Sequences of Operation for HVAC Systems, American Society of Heating, Refrigerating and Air-Conditioning Engineers. Technical report."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"115005","DOI":"10.1016\/j.enbuild.2024.115005","article-title":"Energy performance evaluation of the ASHRAE Guideline 36 control and reinforcement learning\u2013based control using field measurements","volume":"325","author":"Yoon","year":"2024","journal-title":"Energy Build."},{"key":"ref_32","unstructured":"Mulayim, O.B., Pergantis, E.N., Premer, L.D.R., Chen, B., Qu, G., Kircher, K.J., and Berg\u00e9s, M. (2025). Comparative Field Deployment of Reinforcement Learning and Model Predictive Control for Residential HVAC. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"126091","DOI":"10.1016\/j.apenergy.2025.126091","article-title":"From theory to practice: A critical review of model predictive control field implementations in the built environment","volume":"393","author":"Saloux","year":"2025","journal-title":"Appl. Energy"},{"key":"ref_34","first-page":"1047","article-title":"Sample-efficient Cross-Entropy Method for Real-time Planning","volume":"Volume 155","author":"Pinneri","year":"2021","journal-title":"Proceedings of the Conference on Robot Learning (CoRL)"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Guillen-Perez, A., Naug, A., Gundecha, V., Ghorbanpour, S., Gutierrez, R.L., Babu, A.R., Salim, M., Banerjee, S., Essink, E.H.O., and Fay, D. (2025). DCcluster-Opt: Benchmarking Dynamic Multi-Objective Optimization for Geo-Distributed Data Center Workloads. arXiv.","DOI":"10.52202\/085713-0172"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/16\/5031\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,10]],"date-time":"2026-08-10T09:35:17Z","timestamp":1786354517000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/16\/5031"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,7]]},"references-count":35,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2026,8]]}},"alternative-id":["s26165031"],"URL":"https:\/\/doi.org\/10.3390\/s26165031","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,7]]}}}