{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,24]],"date-time":"2026-03-24T22:40:34Z","timestamp":1774392034276,"version":"3.50.1"},"reference-count":32,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2022,12,5]],"date-time":"2022-12-05T00:00:00Z","timestamp":1670198400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Brazilian Informatics Law","award":["HP 052-21"],"award-info":[{"award-number":["HP 052-21"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Head-mounted displays are virtual reality devices that may be equipped with sensors and cameras to measure a patient\u2019s heart rate through facial regions. Heart rate is an essential body signal that can be used to remotely monitor users in a variety of situations. There is currently no study that predicts heart rate using only highlighted facial regions; thus, an adaptation is required for beats per minute predictions. Likewise, there are no datasets containing only the eye and lower face regions, necessitating the development of a simulation mechanism. This work aims to remotely estimate heart rate from facial regions that can be captured by the cameras of a head-mounted display using state-of-the-art EVM-CNN and Meta-rPPG techniques. We developed a region of interest extractor to simulate a dataset from a head-mounted display device using stabilizer and video magnification techniques. Then, we combined support vector machine and FaceMash to determine the regions of interest and adapted photoplethysmography and beats per minute signal predictions to work with the other techniques. We observed an improvement of 188.88% for the EVM and 55.93% for the Meta-rPPG. In addition, both models were able to predict heart rate using only facial regions as input. Moreover, the adapted technique Meta-rPPG outperformed the original work, whereas the EVM adaptation produced comparable results for the photoplethysmography signal.<\/jats:p>","DOI":"10.3390\/s22239486","type":"journal-article","created":{"date-parts":[[2022,12,5]],"date-time":"2022-12-05T08:10:57Z","timestamp":1670227857000},"page":"9486","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Remote Heart Rate Prediction in Virtual Reality Head-Mounted Displays Using Machine Learning Techniques"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2457-9064","authenticated-orcid":false,"given":"Tiago Palma","family":"Pagano","sequence":"first","affiliation":[{"name":"Computational Modeling Department, SENAI CIMATEC University Center, Salvador 41650-010, Bahia, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5949-8515","authenticated-orcid":false,"given":"Lucas Lisboa","family":"dos Santos","sequence":"additional","affiliation":[{"name":"Computational Modeling Department, SENAI CIMATEC University Center, Salvador 41650-010, Bahia, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2967-8766","authenticated-orcid":false,"given":"Victor Rocha","family":"Santos","sequence":"additional","affiliation":[{"name":"Computational Modeling Department, SENAI CIMATEC University Center, Salvador 41650-010, Bahia, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9846-6308","authenticated-orcid":false,"given":"Paulo H. Miranda","family":"S\u00e1","sequence":"additional","affiliation":[{"name":"Computational Modeling Department, SENAI CIMATEC University Center, Salvador 41650-010, Bahia, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9297-3605","authenticated-orcid":false,"given":"Yasmin da Silva","family":"Bonfim","sequence":"additional","affiliation":[{"name":"Computational Modeling Department, SENAI CIMATEC University Center, Salvador 41650-010, Bahia, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4747-3134","authenticated-orcid":false,"given":"Jos\u00e9 Vinicius Dantas","family":"Paranhos","sequence":"additional","affiliation":[{"name":"Computational Modeling Department, SENAI CIMATEC University Center, Salvador 41650-010, Bahia, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9248-3596","authenticated-orcid":false,"given":"Lucas Lemos","family":"Ortega","sequence":"additional","affiliation":[{"name":"Computational Modeling Department, SENAI CIMATEC University Center, Salvador 41650-010, Bahia, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9393-857X","authenticated-orcid":false,"given":"Lian F. Santana","family":"Nascimento","sequence":"additional","affiliation":[{"name":"Computational Modeling Department, SENAI CIMATEC University Center, Salvador 41650-010, Bahia, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alexandre","family":"Santos","sequence":"additional","affiliation":[{"name":"HP Inc. Brazil R&D, Porto Alegre 90619-900, Rio Grande do Sul, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3924-7329","authenticated-orcid":false,"given":"Maikel Maciel","family":"R\u00f6nnau","sequence":"additional","affiliation":[{"name":"HP Inc. Brazil R&D, Porto Alegre 90619-900, Rio Grande do Sul, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6505-6636","authenticated-orcid":false,"given":"Ingrid","family":"Winkler","sequence":"additional","affiliation":[{"name":"Department of Management and Industrial Technology, SENAI CIMATEC University Center, Salvador 41650-010, Bahia, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2219-0290","authenticated-orcid":false,"given":"Erick G. Sperandio","family":"Nascimento","sequence":"additional","affiliation":[{"name":"Department of Management and Industrial Technology, SENAI CIMATEC University Center, Salvador 41650-010, Bahia, Brazil"},{"name":"Faculty of Engineering and Physical Sciences, School of Computer Science and Electronic Engineering, Surrey Institute for People-Centred AI, University of Surrey, Guildford GU2 7XH, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,12,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"101634","DOI":"10.1016\/j.bspc.2019.101634","article-title":"Non-contact-based driver\u2019s cognitive load classification using physiological and vehicular parameters","volume":"55","author":"Rahman","year":"2020","journal-title":"Biomed. Signal Process. Control"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"639","DOI":"10.3389\/fphys.2018.00639","article-title":"Heart rate monitoring in team sports\u2014A conceptual framework for contextualizing heart rate measures for training and recovery prescription","volume":"9","author":"Schneider","year":"2018","journal-title":"Front. Physiol."},{"key":"ref_3","first-page":"9","article-title":"Differences in Heart Rate and Galvanic Skin Response among Nurses Working in Critical and Non-Critical Care Units","volume":"12","author":"Sharma","year":"2018","journal-title":"J. Clin. Diagn. Res."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"655","DOI":"10.1080\/15614263.2019.1666006","article-title":"Towards systematic and objective evaluation of police officer performance in stressful situations","volume":"21","author":"Bertilsson","year":"2020","journal-title":"Police Pract. Res."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Siddharth, S., and Trivedi, M.M. (2020). On Assessing Driver Awareness of Situational Criticalities: Multi-modal Bio-Sensing and Vision-Based Analysis, Evaluations, and Insights. Brain Sci., 10.","DOI":"10.3390\/brainsci10010046"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Yu, W., Ding, S., Yue, Z., and Yang, S. (2020, January 9\u201311). Emotion Recognition from Facial Expressions and Contactless Heart Rate Using Knowledge Graph. Proceedings of the 2020 IEEE International Conference on Knowledge Graph (ICKG), Nanjing, China.","DOI":"10.1109\/ICBK50248.2020.00019"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1097\/FBP.0000000000000383","article-title":"Heart-rate variability: A biomarker to study the influence of nutrition on physiological and psychological health?","volume":"29","author":"Young","year":"2018","journal-title":"Behav. Pharmacol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1770","DOI":"10.1111\/psyp.12915","article-title":"Resting heart rate, guilt, and sympathy: A developmental psychophysiological study of physical aggression","volume":"54","author":"Colasante","year":"2017","journal-title":"Psychophysiology"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Huang, B., Chang, C.M., Lin, C.L., Chen, W., Juang, C.F., and Wu, X. (2020, January 9\u201313). Visual Heart Rate Estimation from Facial Video Based on CNN. Proceedings of the 2020 15th IEEE Conference on Industrial Electronics and Applications (ICIEA), Virtual.","DOI":"10.1109\/ICIEA48937.2020.9248356"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Hsu, G., Ambikapathi, A., and Chen, M. (2017, January 1\u20134). Deep learning with time-frequency representation for pulse estimation from facial videos. Proceedings of the 2017 IEEE International Joint Conference on Biometrics (IJCB), Denver, CO, USA.","DOI":"10.1109\/BTAS.2017.8272721"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"155","DOI":"10.1016\/j.neucom.2020.07.012","article-title":"A deep learning framework for heart rate estimation from facial videos","volume":"417","author":"Hsu","year":"2020","journal-title":"Neurocomputing"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"R1","DOI":"10.1088\/0967-3334\/28\/3\/R01","article-title":"Photoplethysmography and its application in clinical physiological measurement","volume":"28","author":"Allen","year":"2007","journal-title":"Physiol. Meas."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"11896","DOI":"10.1109\/ACCESS.2020.2964794","article-title":"Non-Contact Emotion Recognition Combining Heart Rate and Facial Expression for Interactive Gaming Environments","volume":"8","author":"Du","year":"2020","journal-title":"IEEE Access"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Szankin, M., Kwasniewska, A., Sirlapu, T., Wang, M., Ruminski, J., Nicolas, R., and Bartscherer, M. (2018, January 18\u201321). Long Distance Vital Signs Monitoring with Person Identification for Smart Home Solutions. Proceedings of the 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu, HI, USA.","DOI":"10.1109\/EMBC.2018.8512509"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"7411","DOI":"10.1109\/TIM.2020.2984168","article-title":"Heart rate estimation from facial videos using a spatiotemporal representation with convolutional neural networks","volume":"69","author":"Song","year":"2020","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1016\/j.compbiomed.2014.02.012","article-title":"Current methods in electrocardiogram characterization","volume":"48","author":"Martis","year":"2014","journal-title":"Comput. Biol. Med."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Scherpf, M., Ernst, H., Malberg, H., and Schmidt, M. (2020, January 13\u201316). DeepPerfusion: Camera-based Blood Volume Pulse Extraction Using a 3D Convolutional Neural Network. Proceedings of the 2020 Computing in Cardiology, Rimini, Italy.","DOI":"10.22489\/CinC.2020.388"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Pagano, T.P., Santos, V.R., Bonfim, Y.d.S., Paranhos, J.V.D., Ortega, L.L., S\u00e1, P.H.M., Nascimento, L.F.S., Winkler, I., and Nascimento, E.G.S. (2022). Machine Learning Models and Videos of Facial Regions for Estimating Heart Rate: A Review on Patents, Datasets, and Literature. Electronics, 11.","DOI":"10.3390\/electronics11091473"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"33","DOI":"10.3389\/fbioe.2018.00033","article-title":"A Comparative Survey of Methods for Remote Heart Rate Detection from Frontal Face Videos","volume":"6","author":"Wang","year":"2018","journal-title":"Front. Bioeng. Biotechnol."},{"key":"ref_20","unstructured":"Goodfellow, I., Bengio, Y., Courville, A., and Bengio, Y. (2016). Deep Learning, MIT Press."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1016\/j.visinf.2020.01.001","article-title":"Enhancing the functionality of augmented reality using deep learning, semantic web and knowledge graphs: A review","volume":"4","author":"Lampropoulos","year":"2020","journal-title":"Vis. Inform."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2100264","DOI":"10.1002\/minf.202100264","article-title":"An AI-based Prediction Model for Drug-drug Interactions in Osteoporosis and Paget\u2019s Diseases from SMILES","volume":"41","author":"Hung","year":"2022","journal-title":"Mol. Inform."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"e4792","DOI":"10.1002\/nbm.4792","article-title":"Molecular subtype classification of low-grade gliomas using magnetic resonance imaging-based radiomics and machine learning","volume":"35","author":"Lam","year":"2022","journal-title":"NMR Biomed."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Furtado, A., Andrade, L., Frias, D., Maia, T., Badar\u00f3, R., and Nascimento, E.G.S. (2022). Deep Learning Applied to Chest Radiograph Classification\u2014A COVID-19 Pneumonia Experience. Appl. Sci., 12.","DOI":"10.3390\/app12083712"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Furtado, A., da Purifica\u00e7\u00e3o, C.A.C., Badar\u00f3, R., and Nascimento, E.G.S. (2022). A Light Deep Learning Algorithm for CT Diagnosis of COVID-19 Pneumonia. Diagnostics, 12.","DOI":"10.3390\/diagnostics12071527"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1778","DOI":"10.1109\/TMM.2018.2883866","article-title":"EVM-CNN: Real-Time Contactless Heart Rate Estimation from Facial Video","volume":"21","author":"Qiu","year":"2019","journal-title":"IEEE Trans. Multimed."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Lee, E., Chen, E., and Lee, C.Y. (2020, January 23\u201328). Meta-rPPG: Remote Heart Rate Estimation Using a Transductive Meta-Learner. Proceedings of the European Conference on Computer Vision (ECCV), Glasgow, UK.","DOI":"10.1007\/978-3-030-58583-9_24"},{"key":"ref_28","unstructured":"Pagano, T.P., Santos, V.R., Bonfim, Y.d.S., Paranhos, J.V.D., Ortega, L.L., S\u00e1, P.H.M., Nascimento, L.F.S., Winkler, I., and Nascimento, E.G.S. (2022, October 26). Remote Heart Machine Learning. Available online: https:\/\/github.com\/CRIA-CIMATEC\/hpvialab\/tree\/main\/remote-heart-machine-learning."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"3589","DOI":"10.1109\/TITS.2020.3038317","article-title":"Near-Infrared Imaging Photoplethysmography during Driving","volume":"23","author":"Nowara","year":"2020","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.patrec.2017.10.017","article-title":"Unsupervised skin tissue segmentation for remote photoplethysmography","volume":"124","author":"Bobbia","year":"2019","journal-title":"Pattern Recognit. Lett."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Ferrari, V., Hebert, M., Sminchisescu, C., and Weiss, Y. Learning-Based Video Motion Magnification. Proceedings of the Computer Vision\u2014ECCV 2018.","DOI":"10.1007\/978-3-030-01264-9"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Montero, A.S., Lang, J., and Lagani\u00e8re, R. (2015, January 7\u201313). Scalable Kernel Correlation Filter with Sparse Feature Integration. Proceedings of the 2015 IEEE International Conference on Computer Vision Workshop (ICCVW), Santiago, Chile.","DOI":"10.1109\/ICCVW.2015.80"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/23\/9486\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:33:58Z","timestamp":1760146438000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/23\/9486"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,5]]},"references-count":32,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["s22239486"],"URL":"https:\/\/doi.org\/10.3390\/s22239486","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,5]]}}}