{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,15]],"date-time":"2026-01-15T06:11:15Z","timestamp":1768457475771,"version":"3.49.0"},"reference-count":42,"publisher":"MDPI AG","issue":"24","license":[{"start":{"date-parts":[[2020,12,8]],"date-time":"2020-12-08T00:00:00Z","timestamp":1607385600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Beijing Education Science Planning Project","award":["CADA18069"],"award-info":[{"award-number":["CADA18069"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Research shows that physiological signals can provide objective data support for the analysis of human emotions. At present, non-contact heart rate data have been employed in the research of medicine, intelligent transportation, smart education, etc. However, it is hard to detect heart rate data using non-contact traditional methods during head rotation, especially when face information is missing in scenarios such as online teaching\/learning. Traditional remote photoplethysmography (rPPG) methods require a static, full frontal face within a fixed distance for heart rate detection. These strict requirements make it impractical to measure heart rate data in real-world scenarios, as a lot of videos only partially record the subjects\u2019 face information, such as profile, too small distance, and wearing a mask. The current algorithm aims to solve the problem of head deflections between 30 degrees and 45 degrees by employing a symmetry substitution method, which can replace the undetected region of interest (ROI) with the detectable one. When face information is partially missing, our algorithm uses face\u2013eye location to determine ROI. The results show that the method in this paper can solve certain practical problems related to heart rate detection, with a root mean square error (RMSE) under 7.64 bpm.<\/jats:p>","DOI":"10.3390\/s20247021","type":"journal-article","created":{"date-parts":[[2020,12,8]],"date-time":"2020-12-08T09:17:04Z","timestamp":1607419024000},"page":"7021","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Non-Contact Heart Rate Detection When Face Information Is Missing during Online Learning"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8966-1184","authenticated-orcid":false,"given":"Kun","family":"Zheng","sequence":"first","affiliation":[{"name":"Faculty of Information Technology, Beijing University of Technology, Beijing 100224, China"}]},{"given":"Kangyi","family":"Ci","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology, Beijing University of Technology, Beijing 100224, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7867-0077","authenticated-orcid":false,"given":"Jinling","family":"Cui","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology, Beijing University of Technology, Beijing 100224, China"}]},{"given":"Jiangping","family":"Kong","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology, Beijing University of Technology, Beijing 100224, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8461-6667","authenticated-orcid":false,"given":"Jing","family":"Zhou","sequence":"additional","affiliation":[{"name":"College of Continuing Education, Beijing University of Technology, Beijing 100224, China"}]}],"member":"1968","published-online":{"date-parts":[[2020,12,8]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1207\/S15326985EP3702_4","article-title":"Academic Emotions in Students\u2019 Self-Regulated Learning and Achievement: A Program of Qualitative and Quantitative Research","volume":"37","author":"Pekrun","year":"2002","journal-title":"Educ. Psychol."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Chettupuzhakkaran, P., and Sindhu, N. (2018, January 11\u201313). Emotion Recognition from Physiological Signals Using Time-Frequency Analysis Methods. Proceedings of the 2018 International Conference on Emerging Trends and Innovations in Engineering and Technological Research (ICETIETR), Ernakulam, India.","DOI":"10.1109\/ICETIETR.2018.8529145"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Joesph, C., Rajeswari, A., Premalatha, B., and Balapriya, C. (2020, January 20\u201324). Implementation of physiological signal based emotion recognition algorithm. Proceedings of the 2020 IEEE 36th International Conference on Data Engineering (ICDE), Dallas, TX, USA.","DOI":"10.1109\/ICDE48307.2020.9153878"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1525\/abt.2015.77.6.413","article-title":"The Experimental Detection of an Emotional Response to the Idea of Evolution","volume":"77","author":"Bland","year":"2015","journal-title":"Am. Biol. Teach."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"192","DOI":"10.1108\/IJILT-11-2018-0127","article-title":"Assessing the emotional impact of virtual reality-based teacher training","volume":"36","author":"Stavroulia","year":"2019","journal-title":"Int. J. Inf. Learn. Technol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/s11423-016-9470-5","article-title":"Genetics with Jean: The design, development and evaluation of an affective tutoring system","volume":"65","author":"Thompson","year":"2016","journal-title":"Educ. Technol. Res. Dev."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"21434","DOI":"10.1364\/OE.16.021434","article-title":"Remote plethysmographic imaging using ambient light","volume":"16","author":"Verkruysse","year":"2008","journal-title":"Opt. Express"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"618","DOI":"10.1109\/JSEN.2014.2347397","article-title":"Image Sensor-Based Heart Rate Evaluation from Face Reflectance Using Hilbert\u2013Huang Transform","volume":"15","author":"Chen","year":"2015","journal-title":"IEEE Sens. J."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1351","DOI":"10.1109\/JSEN.2015.2500032","article-title":"Face-Based Heart Rate Signal Decomposition and Evaluation Using Multiple Linear Regression","volume":"16","author":"Lin","year":"2016","journal-title":"IEEE Sens. J."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"8420","DOI":"10.1109\/JSEN.2018.2872623","article-title":"A Novel EEMD-CCA Approach to Removing Muscle Artifacts for Pervasive EEG","volume":"19","author":"Chen","year":"2018","journal-title":"IEEE Sens. J."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1422","DOI":"10.1109\/JBHI.2016.2615472","article-title":"Illumination Variation-Resistant Video-Based Heart Rate Measurement Using Joint Blind Source Separation and Ensemble Empirical Mode Decomposition","volume":"21","author":"Cheng","year":"2017","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1049\/el.2016.3611","article-title":"Illumination variation interference suppression in remote PPG using PLS and MEMD","volume":"53","author":"Xu","year":"2017","journal-title":"Electron. Lett."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"10762","DOI":"10.1364\/OE.18.010762","article-title":"Non-contact, automated cardiac pulse measurements using video imaging and blind source separation","volume":"18","author":"Poh","year":"2010","journal-title":"Opt. Express"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2878","DOI":"10.1109\/TBME.2013.2266196","article-title":"Robust Pulse Rate from Chrominance-Based rPPG","volume":"60","author":"Jeanne","year":"2013","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"568","DOI":"10.1016\/j.bspc.2013.05.010","article-title":"Continuous wavelet filtering on webcam photoplethysmographic signals to remotely assess the instantaneous heart rate","volume":"8","author":"Bousefsaf","year":"2013","journal-title":"Biomed. Signal Process. Control."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"873","DOI":"10.1364\/BOE.9.000873","article-title":"Bounded Kalman filter method for motion-robust, non-contact heart rate estimation","volume":"9","author":"Prakash","year":"2018","journal-title":"Biomed. Opt. Express"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"McDuff, D.J., Blackford, E.B., and Estepp, J.R. (June, January 30). The Impact of Video Compression on Remote Cardiac Pulse Measurement Using Imaging Photoplethysmography. Proceedings of the 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017), Washington, DC, USA.","DOI":"10.1109\/FG.2017.17"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Zhao, C., Lin, C.-L., Chen, W., and Li, Z. (2018, January 18\u201322). A Novel Framework for Remote Photoplethysmography Pulse Extraction on Compressed Videos. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPRW.2018.00177"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"4537","DOI":"10.1109\/JSEN.2019.2899102","article-title":"Physiological Signal Preserving Video Compression for Remote Photoplethysmography","volume":"19","author":"Zhao","year":"2019","journal-title":"IEEE Sens. J."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"366","DOI":"10.1016\/j.bspc.2018.09.007","article-title":"Robust remote heart rate estimation from multiple asynchronous noisy channels using autoregressive model with Kalman filter","volume":"47","author":"Nooralishahi","year":"2019","journal-title":"Biomed. Signal Process. Control."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1007\/s11760-018-01409-w","article-title":"Motion-resistant heart rate measurement from face videos using patch-based fusion","volume":"13","author":"Yang","year":"2019","journal-title":"Signal Image Video Process."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"26747","DOI":"10.1007\/s11042-019-07849-x","article-title":"Motion-tolerant heart rate estimation from face videos using derivative filter","volume":"78","author":"Yang","year":"2019","journal-title":"Multimed. Tools Appl."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Bobbia, S., Luguern, D., Benezeth, Y., Nakamura, K., Gomez, R., and Dubois, J. (2018, January 18\u201322). Real-Time Temporal Superpixels for Unsupervised Remote Photoplethysmography. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPRW.2018.00182"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2106","DOI":"10.3758\/s13428-019-01256-8","article-title":"An open-source remote heart rate imaging method with practical apparatus and algorithms","volume":"51","author":"Naber","year":"2019","journal-title":"Behav. Res. Methods"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"085007","DOI":"10.1088\/1361-6579\/ab2c9f","article-title":"Robust heart-rate estimation from facial videos using Project_ICA","volume":"40","author":"Qi","year":"2019","journal-title":"Physiol. Meas."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"76513","DOI":"10.1109\/ACCESS.2019.2922304","article-title":"Optimizing Remote Photoplethysmography Using Adaptive Skin Segmentation for Real-Time Heart Rate Monitoring","volume":"7","author":"Fouad","year":"2019","journal-title":"IEEE Access"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1778","DOI":"10.1109\/TMM.2018.2883866","article-title":"EVM-CNN: Real-Time Contactless Heart Rate Estimation From Facial Video","volume":"21","author":"Qiu","year":"2018","journal-title":"IEEE Trans. Multimed."},{"key":"ref_28","first-page":"27","article-title":"Research on non-contact physiological parameter measurement algorithm based on IPPG","volume":"37","author":"Rong","year":"2018","journal-title":"Biomed. Eng. Res."},{"key":"ref_29","unstructured":"Dalal, N., and Triggs, B. (2005, January 20\u201325). Histograms of Oriented Gradients for Human Detection. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201905), San Diego, CA, USA."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1023\/B:VISI.0000013087.49260.fb","article-title":"Robust Real-Time Face Detection","volume":"57","author":"Viola","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1499","DOI":"10.1109\/LSP.2016.2603342","article-title":"Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks","volume":"23","author":"Zhang","year":"2016","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Kowalski, M., Naruniec, J., and Trzcinski, T. (2017, January 21\u201326). Deep Alignment Network: A Convolutional Neural Network for Robust Face Alignment. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, HI, USA.","DOI":"10.1109\/CVPRW.2017.254"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1109\/T-AFFC.2011.25","article-title":"A Multimodal Database for Affect Recognition and Implicit Tagging","volume":"3","author":"Soleymani","year":"2012","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/T-AFFC.2011.15","article-title":"DEAP: A Database for Emotion Analysis Using Physiological Signals","volume":"3","author":"Koelstra","year":"2011","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Tulyakov, S., Alameda-Pineda, X., Ricci, E., Yin, L., Cohn, J.F., and Sebe, N. (2016, January 27\u201330). Self-Adaptive Matrix Completion for Heart Rate Estimation from Face Videos under Realistic Conditions. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.263"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Stricker, R., Muller, S., and Gross, H.-M. (2014, January 25\u201329). Non-contact video-based pulse rate measurement on a mobile service robot. Proceedings of the the 23rd IEEE International Symposium on Robot and Human Interactive Communication, Edinburgh, UK.","DOI":"10.1109\/ROMAN.2014.6926392"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Li, X., Alikhani, I., Shi, J., Seppanen, T., Junttila, J., Majamaa-Voltti, K., Tulppo, M., and Zhao, G. (2018, January 15\u201319). The OBF Database: A Large Face Video Database for Remote Physiological Signal Measurement and Atrial Fibrillation Detection. Proceedings of the 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi\u2019an, China.","DOI":"10.1109\/FG.2018.00043"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"2409","DOI":"10.1109\/TIP.2019.2947204","article-title":"RhythmNet: End-to-End Heart Rate Estimation from Face via Spatial-Temporal Representation","volume":"29","author":"Niu","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1109\/TBME.2010.2086456","article-title":"Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam","volume":"58","author":"Poh","year":"2010","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Lam, A., and Kuno, Y. (2015, January 7\u201313). Robust Heart Rate Measurement from Video Using Select Random Patches. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.415"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1479","DOI":"10.1109\/TBME.2016.2609282","article-title":"Algorithmic Principles of Remote PPG","volume":"64","author":"Wang","year":"2017","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"702","DOI":"10.1007\/978-3-642-33765-9_50","article-title":"Exploiting the Circulant Structure of Tracking-by-Detection with Kernels","volume":"7575","author":"Henriques","year":"2012","journal-title":"Lect. Notes Comput. Sci."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/24\/7021\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:42:25Z","timestamp":1760179345000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/24\/7021"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,8]]},"references-count":42,"journal-issue":{"issue":"24","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["s20247021"],"URL":"https:\/\/doi.org\/10.3390\/s20247021","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12,8]]}}}