{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:45:47Z","timestamp":1760143547459,"version":"build-2065373602"},"reference-count":52,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2024,2,14]],"date-time":"2024-02-14T00:00:00Z","timestamp":1707868800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Science Foundation of China","award":["62073201","62173216"],"award-info":[{"award-number":["62073201","62173216"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The construction sector is responsible for almost 30% of the world\u2019s total energy consumption, with a significant portion of this energy being used by heating, ventilation and air-conditioning (HVAC) systems to ensure people\u2019s thermal comfort. In practical applications, the conventional approach to HVAC management in buildings typically involves the manual control of temperature setpoints by facility operators. Nevertheless, the implementation of real-time alterations that are based on the thermal comfort levels of humans inside a building has the potential to dramatically improve the energy efficiency of the structure. Therefore, we propose a model for non-intrusive, dynamic inference of occupant thermal comfort based on building indoor surveillance camera data. It is based on a two-stream transformer-augmented adaptive graph convolutional network to identify people\u2019s heat-related adaptive behaviors. The transformer specifically strengthens the original adaptive graph convolution network module, resulting in further improvement to the accuracy of the detection of thermal adaptation behavior. The experiment is conducted on a dataset including 16 distinct temperature adaption behaviors. The findings indicate that the suggested strategy significantly improves the behavior recognition accuracy of the proposed model to 96.56%. The proposed model provides the possibility to realize energy savings and emission reductions in intelligent buildings and dynamic decision making in energy management systems.<\/jats:p>","DOI":"10.3390\/s24041219","type":"journal-article","created":{"date-parts":[[2024,2,14]],"date-time":"2024-02-14T09:30:18Z","timestamp":1707903018000},"page":"1219","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Thermal-Adaptation-Behavior-Based Thermal Sensation Evaluation Model with Surveillance Cameras"],"prefix":"10.3390","volume":"24","author":[{"given":"Yu","family":"Wang","sequence":"first","affiliation":[{"name":"School of Information Science and Engineering, Shandong Normal University, Jinan 250358, China"}]},{"given":"Wenjun","family":"Duan","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Shandong Jianzhu University, Jinan 250101, China"}]},{"given":"Junqing","family":"Li","sequence":"additional","affiliation":[{"name":"Sohool of Computer Science, Liaocheng University, Liaocheng 252000, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8433-2373","authenticated-orcid":false,"given":"Dongdong","family":"Shen","sequence":"additional","affiliation":[{"name":"School of Information Science and Engineering, Shandong Normal University, Jinan 250358, China"}]},{"given":"Peiyong","family":"Duan","sequence":"additional","affiliation":[{"name":"School of Information Science and Engineering, Shandong Normal University, Jinan 250358, China"}]}],"member":"1968","published-online":{"date-parts":[[2024,2,14]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1016\/j.jclepro.2015.05.139","article-title":"Energy consumption and efficiency in buildings: Current status and future trends","volume":"109","author":"Allouhi","year":"2015","journal-title":"J. Clean. Prod."},{"key":"ref_2","unstructured":"United Nations Environment Programme (2020). 2020 Global Status Report for Buildings and Construction: Towards a Zero-Emission, Efficient and Resilient Buildings and Construction Sector, Global Alliance for Buildings and Construction."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"103406","DOI":"10.1016\/j.jobe.2021.103406","article-title":"Building energy consumption prediction for residential buildings using deep learning and other machine learning techniques","volume":"45","author":"Alaka","year":"2022","journal-title":"J. Build. Eng."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"100465","DOI":"10.1016\/j.mtphys.2021.100465","article-title":"Personal thermal management techniques for thermal comfort and building energy saving","volume":"20","author":"Ma","year":"2021","journal-title":"Mater. Today Phys."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"110332","DOI":"10.1016\/j.buildenv.2023.110332","article-title":"A novel occupant-centric stratum ventilation system using computer vision: Occupant detection, thermal comfort, air quality, and energy savings","volume":"237","author":"Yang","year":"2023","journal-title":"Build. Environ."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1016\/j.enbuild.2019.05.050","article-title":"Indoor thermal environment optimal control for thermal comfort and energy saving based on online monitoring of thermal sensation","volume":"197","author":"Li","year":"2019","journal-title":"Energy Build."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Lopez, G., Aoki, T., Nkurikiyeyezu, K., and Yokokubo, A. (2020). Model for thermal comfort and energy saving based on individual sensation estimation. arXiv.","DOI":"10.18494\/SAM.2020.2635"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"107432","DOI":"10.1016\/j.buildenv.2020.107432","article-title":"Experimental study of an indoor temperature fuzzy control method for thermal comfort and energy saving using wristband device","volume":"187","author":"Li","year":"2021","journal-title":"Build. Environ."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1323","DOI":"10.1007\/s12273-021-0814-7","article-title":"Comparison study of thermal comfort and energy saving under eight different ventilation modes for space heating","volume":"Volume 15","author":"Kong","year":"2022","journal-title":"Proceedings of the Building Simulation"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"113719","DOI":"10.1016\/j.enbuild.2023.113719","article-title":"A personalized regression model for predicting thermal sensation based on local skin temperature in moderate summer conditions","volume":"301","author":"Qi","year":"2023","journal-title":"Energy Build."},{"key":"ref_11","unstructured":"Fanger, P.O. (1970). THermal Comfort. Analysis and Applications in Environmental Engineering, Danish Technical Press."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1386","DOI":"10.1016\/j.buildenv.2008.04.006","article-title":"Relationship between thermal sensation and comfort in non-uniform and dynamic environments","volume":"44","author":"Zhang","year":"2009","journal-title":"Build. Environ."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Nicol, F., Humphreys, M., and Roaf, S. (2012). Adaptive Thermal Comfort: Principles and Practice, Routledge.","DOI":"10.4324\/9780203123010"},{"key":"ref_14","unstructured":"(2013). Thermal Environmental Conditions for Human Occupancy (Standard No. ANSI\/ASHRAE Standard 55-2013)."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1016\/j.enbuild.2019.01.051","article-title":"Progress in thermal comfort studies in classrooms over last 50 years and way forward","volume":"188","author":"Singh","year":"2019","journal-title":"Energy Build."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"106281.1","DOI":"10.1016\/j.buildenv.2019.106281","article-title":"Personal thermal comfort models with wearable sensors","volume":"162","author":"Liu","year":"2019","journal-title":"Build. Environ."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1016\/j.enbuild.2018.09.041","article-title":"The uncertainty of subjective thermal comfort measurement","volume":"181","author":"Wang","year":"2018","journal-title":"Energy Build."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"340","DOI":"10.1016\/j.enbuild.2019.06.007","article-title":"NIDL: A pilot study of contactless measurement of skin temperature for intelligent building","volume":"198","author":"Cheng","year":"2019","journal-title":"Energy Build."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"107130","DOI":"10.1016\/j.buildenv.2020.107130","article-title":"Evaluation of comfort in subway stations via electroencephalography measurements in field experiments","volume":"183","author":"Kim","year":"2020","journal-title":"Build. Environ."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1016\/j.enbuild.2018.05.056","article-title":"Experimental research of online monitoring and evaluation method of human thermal sensation in different active states based on wristband device","volume":"173","author":"Li","year":"2018","journal-title":"Energy Build."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"534","DOI":"10.1111\/ina.12644","article-title":"Using electroencephalogram to continuously discriminate feelings of personal thermal comfort between uncomfortably hot and comfortable environments","volume":"30","author":"Wu","year":"2020","journal-title":"Indoor Air"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"108064","DOI":"10.1016\/j.buildenv.2021.108064","article-title":"Review of vision-based occupant information sensing systems for occupant-centric control","volume":"203","author":"Choi","year":"2021","journal-title":"Build. Environ."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"106163","DOI":"10.1016\/j.buildenv.2019.106163","article-title":"Using the contrast within a single face heat map to assess personal thermal comfort","volume":"160","author":"Cosma","year":"2019","journal-title":"Build. Environ."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"829","DOI":"10.1016\/j.apenergy.2018.02.049","article-title":"Personalized thermal comfort inference using RGB video images for distributed HVAC control","volume":"220","author":"Jazizadeh","year":"2018","journal-title":"Appl. Energy"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"113336","DOI":"10.1016\/j.apenergy.2019.113336","article-title":"Robust non-intrusive interpretation of occupant thermal comfort in built environments with low-cost networked thermal cameras","volume":"251","author":"Li","year":"2019","journal-title":"Appl. Energy"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"246","DOI":"10.1016\/j.enbuild.2018.07.025","article-title":"Non-intrusive interpretation of human thermal comfort through analysis of facial infrared thermography","volume":"176","author":"Li","year":"2018","journal-title":"Energy Build."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1063","DOI":"10.1016\/j.enbuild.2017.09.032","article-title":"Real-time human skin temperature analysis using thermal image recognition for thermal comfort assessment","volume":"158","author":"Metzmacher","year":"2018","journal-title":"Energy Build."},{"key":"ref_28","unstructured":"Vesely, M., Cieszczyk, A., Zhao, Y., and Zeiler, W. (2015, January 9\u201311). Low cost infrared array as a thermal comfort sensor. Proceedings of the International Conference CISBAT 2015 Future Buildings and Districts Sustainability from Nano to Urban Scale, LESO-PB, EPFL, Number CONF, Lausanne, Switzerland."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ranjan, J., and Scott, J. (2016, January 12\u201316). ThermalSense: Determining dynamic thermal comfort preferences using thermographic imaging. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Heidelberg, Germany.","DOI":"10.1145\/2971648.2971659"},{"key":"ref_30","unstructured":"Vissers, D., and Zeiler, W. (2012, January 7\u20139). The user as sensor to reach for optimal individual comfort and reduced energy consumption. Proceedings of the PLEA 2012: Towards An Environmentally Responsible Architecture, Lima, Peru."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1016\/j.buildenv.2018.06.052","article-title":"Thermal comfort modeling in transient conditions using real-time local body temperature extraction with a thermographic camera","volume":"143","author":"Cosma","year":"2018","journal-title":"Build. Environ."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Cao, Z., Simon, T., Wei, S.E., and Sheikh, Y. (2017, January 21\u201326). Realtime multi-person 2d pose estimation using part affinity fields. Proceedings of the the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.143"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"372","DOI":"10.1016\/j.buildenv.2018.11.017","article-title":"Machine learning method for real-time non-invasive prediction of individual thermal preference in transient conditions","volume":"148","author":"Cosma","year":"2019","journal-title":"Build. Environ."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.buildenv.2017.05.021","article-title":"A pilot study of online non-invasive measuring technology based on video magnification to determine skin temperature","volume":"121","author":"Cheng","year":"2017","journal-title":"Build. Environ."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Cheng, X., Yang, B., Tan, K., Isaksson, E., Li, L., Hedman, A., Olofsson, T., and Li, H. (2019). A contactless measuring method of skin temperature based on the skin sensitivity index and deep learning. Appl. Sci., 9.","DOI":"10.3390\/app9071375"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1016\/j.buildenv.2018.05.018","article-title":"Vision-based thermal comfort quantification for HVAC control","volume":"142","author":"Jung","year":"2018","journal-title":"Build. Environ."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2185520.2185561","article-title":"Eulerian video magnification for revealing subtle changes in the world","volume":"31","author":"Wu","year":"2012","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"106284.1","DOI":"10.1016\/j.buildenv.2019.106284","article-title":"Real-time and contactless measurements of thermal discomfort based on human poses for energy efficient control of buildings","volume":"162","author":"Yang","year":"2019","journal-title":"Build. Environ."},{"key":"ref_39","unstructured":"Meier, A., Dyer, W., and Graham, C. (2017, January 13\u201315). Using human gestures to control a building\u2019s heating and cooling System. Proceedings of the International Conference on Energy Efficiency in Domestic Appliances and Lighting (EEDAL\u201917), Irvine, CA, USA."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"110989","DOI":"10.1016\/j.enbuild.2021.110989","article-title":"Real-time surveillance-video-based personalized thermal comfort recognition","volume":"244","author":"Duan","year":"2021","journal-title":"Energy Build."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Yan, S., Xiong, Y., and Lin, D. (2018, January 2\u20137). Spatial temporal graph convolutional networks for skeleton-based action recognition. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.12328"},{"key":"ref_42","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017). Attention is all you need. Adv. Neural Inf. Process. Syst., 30."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"103219","DOI":"10.1016\/j.cviu.2021.103219","article-title":"Skeleton-based action recognition via spatial and temporal transformer networks","volume":"208","author":"Plizzari","year":"2021","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Shi, L., Zhang, Y., Cheng, J., and Lu, H. (2019, January 15\u201320). Two-stream adaptive graph convolutional networks for skeleton-based action recognition. Proceedings of the the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01230"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Wang, X., Girshick, R., Gupta, A., and He, K. (2018, January 18\u201323). Non-local neural networks. Proceedings of the the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00813"},{"key":"ref_46","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv."},{"key":"ref_47","unstructured":"Bello, I., Zoph, B., Vaswani, A., Shlens, J., and Le, Q.V. (November, January 27). Attention augmented convolutional networks. Proceedings of the the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_48","unstructured":"He, S., Liao, W., Tavakoli, H.R., Yang, M., Rosenhahn, B., and Pugeault, N. (December, January 30). Image captioning through image transformer. Proceedings of the the Asian Conference on Computer Vision, Kyoto, Japan."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. (2020, January 23\u201328). End-to-end object detection with transformers. Proceedings of the European Conference on Computer Vision, Glasgow, UK.","DOI":"10.1007\/978-3-030-58452-8_13"},{"key":"ref_50","unstructured":"Huang, Z., Wang, X., Huang, L., Huang, C., Wei, Y., and Liu, W. (November, January 27). Ccnet: Criss-cross attention for semantic segmentation. Proceedings of the the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_51","unstructured":"Lee, S., Yu, Y., Kim, G., Breuel, T., Kautz, J., and Song, Y. (2020). Parameter efficient multimodal transformers for video representation learning. arXiv."},{"key":"ref_52","unstructured":"Van den Oord, A., Kalchbrenner, N., Espeholt, L., Vinyals, O., Graves, A., and Kavukcuoglu, K. (2016). Conditional image generation with pixelcnn decoders. Adv. Neural Inf. Process. Syst., 29."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/4\/1219\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T13:59:47Z","timestamp":1760104787000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/4\/1219"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,14]]},"references-count":52,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,2]]}},"alternative-id":["s24041219"],"URL":"https:\/\/doi.org\/10.3390\/s24041219","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2024,2,14]]}}}