{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,15]],"date-time":"2026-10-15T16:29:00Z","timestamp":1792081740155,"version":"4.4.0"},"reference-count":54,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2023,5,7]],"date-time":"2023-05-07T00:00:00Z","timestamp":1683417600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Detecting vital signs by using a contactless camera-based approach can provide several advantages over traditional clinical methods, such as lower financial costs, reduced visit times, increased comfort, and enhanced safety for healthcare professionals. Specifically, Eulerian Video Magnification (EVM) or Remote Photoplethysmography (rPPG) methods can be utilised to remotely estimate heart rate and respiratory rate biomarkers. In this paper two contactless camera-based health monitoring architectures are developed using EVM and rPPG, respectively; to this end, two different CNNs, (Mediapipe\u2019s BlazeFace and FaceMesh) are used to extract suitable regions of interest from incoming video frames. These two methods are implemented and deployed on four off-the-shelf edge devices as well as on a PC and evaluated in terms of latency (in each stage of the application\u2019s pipeline), throughput (FPS), power consumption (Watt), efficiency (throughput\/Watt), and value (throughput\/cost). This work provides important insights about the computational costs and bottlenecks of each method on each hardware platform, as well as which platform to use depending on the target metric. One of our insights shows that the Jetson Xavier NX platform is the best platform in terms of throughput and efficiency, while Raspberry Pi 4 8 GB is the best platform in terms of value.<\/jats:p>","DOI":"10.3390\/s23094550","type":"journal-article","created":{"date-parts":[[2023,5,8]],"date-time":"2023-05-08T02:29:22Z","timestamp":1683512962000},"page":"4550","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["Contactless Camera-Based Heart Rate and Respiratory Rate Monitoring Using AI on Hardware"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2116-785X","authenticated-orcid":false,"given":"Dimitrios","family":"Kolosov","sequence":"first","affiliation":[{"name":"School of Physics, Engineering and Computer Science, University of Hertfordshire, Hatfield AL10 9AB, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vasilios","family":"Kelefouras","sequence":"additional","affiliation":[{"name":"School of Engineering, Computing and Mathematics, University of Plymouth, Plymouth PL4 8AA, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pandelis","family":"Kourtessis","sequence":"additional","affiliation":[{"name":"School of Physics, Engineering and Computer Science, University of Hertfordshire, Hatfield AL10 9AB, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6984-0268","authenticated-orcid":false,"given":"Iosif","family":"Mporas","sequence":"additional","affiliation":[{"name":"School of Physics, Engineering and Computer Science, University of Hertfordshire, Hatfield AL10 9AB, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,5,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Hasan, Z., Dey, E., Ramamurthy, S.R., Roy, N., and Misra, A. (2022, January 20\u201324). Demo: RhythmEdge: Enabling Contactless Heart Rate Estimation on the Edge. Proceedings of the 2022 IEEE International Conference on Smart Computing (SMARTCOMP), Helsinki, Finland.","DOI":"10.1109\/SMARTCOMP55677.2022.00044"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Aubakir, B., Nurimbetov, B., Tursynbek, I., and Varol, H.A. (2016, January 16\u201320). Vital sign monitoring utilizing Eulerian video magnification and thermography. Proceedings of the 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA.","DOI":"10.1109\/EMBC.2016.7591489"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"4711","DOI":"10.1109\/ACCESS.2017.2678521","article-title":"Heart Rate Variability Extraction From Videos Signals: ICA vs. EVM Comparison","volume":"5","author":"Alghoul","year":"2017","journal-title":"IEEE Access"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Gambi, E., Agostinelli, A., Belli, A., Burattini, L., Cippitelli, E., Fioretti, S., Pierleoni, P., Ricciuti, M., Sbrollini, A., and Spinsante, S. (2017). Heart Rate Detection Using Microsoft Kinect: Validation and Comparison to Wearable Devices. Sensors, 17.","DOI":"10.3390\/s17081776"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2516","DOI":"10.1109\/TIM.2017.2684518","article-title":"Adaptive Eulerian Video Processing of Thermal Video: An Experimental Analysis","volume":"66","author":"Bennett","year":"2017","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1515\/mms-2016-0040","article-title":"Reliability of Pulse Measurements in Videoplethysmography","volume":"23","year":"2016","journal-title":"Metrol. Meas. Syst."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"04NT01","DOI":"10.1088\/1361-6579\/ab840a","article-title":"Feasible assessment of recovery and cardiovascular health: Accuracy of nocturnal HR and HRV assessed via ring PPG in comparison to medical grade ECG","volume":"41","author":"Kinnunen","year":"2020","journal-title":"Physiol. Meas."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Madhav, K.V., Ram, M.R., Krishna, E.H., and Reddy, K.N. (December, January 30). Estimation of respiratory rate from principal components of photoplethysmographic signals. Proceedings of the IEEE EMBS Conference on Biomedical Engineering and Sciences, IECBES 2010, Kuala Lumpur, Malaysia.","DOI":"10.1109\/IECBES.2010.5742251"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1946","DOI":"10.1109\/TBME.2013.2246160","article-title":"Multiparameter Respiratory Rate Estimation From the Photoplethysmogram","volume":"60","author":"Karlen","year":"2013","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Ni, A., Azarang, A., and Kehtarnavaz, N. (2021). A Review of Deep Learning-Based Contactless Heart Rate Measurement Methods. Sensors, 21.","DOI":"10.3390\/s21113719"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Dosso, Y.S., Bekele, A., and Green, J.R. (2018, January 11\u201313). Eulerian Magnification of Multi-Modal RGB-D Video for Heart Rate Estimation. Proceedings of the 2018 IEEE International Symposium on Medical Measurements and Applications (MeMeA), Rome, Italy.","DOI":"10.1109\/MeMeA.2018.8438741"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2185520.2185561","article-title":"Eulerian video magnification for revealing subtle changes in the world","volume":"31","author":"Wu","year":"2012","journal-title":"ACM Trans. Graph."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"17940","DOI":"10.1038\/s41598-020-74954-5","article-title":"Automated video-based heart rate tracking for the anesthetized and behaving monkey","volume":"10","author":"Froesel","year":"2020","journal-title":"Sci. Rep."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"104378","DOI":"10.1016\/j.ijmedinf.2021.104378","article-title":"An evaluation of a video magnification-based system for respiratory rate monitoring in an acute mental health setting","volume":"148","author":"Laurie","year":"2021","journal-title":"Int. J. Med. Inform."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zhang, J., Zhang, K., Yang, X., and Wen, C. (2020, January 22\u201324). Heart rate measurement based on video acceleration magnification. Proceedings of the 2020 Chinese Control And Decision Conference (CCDC), Hefei, China.","DOI":"10.1109\/CCDC49329.2020.9164451"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Liu, L., Luo, J., Zhang, J., and Chen, X. (2014, January 14\u201316). Enhanced Eulerian video magnification. Proceedings of the 2014 7th International Congress on Image and Signal Processing, CISP 2014, Dalian, China.","DOI":"10.1109\/CISP.2014.7003748"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Bennett, S.L., Goubran, R., and Knoefel, F. (2016, January 15\u201318). Adaptive eulerian video magnification methods to extract heart rate from thermal video. Proceedings of the 2016 IEEE International Symposium on Medical Measurements and Applications (MeMeA), Benevento, Italy.","DOI":"10.1109\/MeMeA.2016.7533818"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1038\/s41746-021-00462-z","article-title":"Evaluation of biases in remote photoplethysmography methods","volume":"4","author":"Dasari","year":"2021","journal-title":"NPJ Digit. Med."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Premkumar, S., and Hemanth, D.J. (2022). Intelligent Remote Photoplethysmography-Based Methods for Heart Rate Estimation from Face Videos: A Survey. Informatics, 9.","DOI":"10.3390\/informatics9030057"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"van Es, V.A.A., Lopata, R.G.P., Scilingo, E.P., and Nardelli, M. (2023). Contactless Cardiovascular Assessment by Imaging Photoplethysmography: A Comparison with Wearable Monitoring. Sensors, 23.","DOI":"10.3390\/s23031505"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Haugg, F., Elgendi, M., and Menon, C. (2022). Effectiveness of Remote PPG Construction Methods: A Preliminary Analysis. Bioengineering, 9.","DOI":"10.3390\/bioengineering9100485"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"21434","DOI":"10.1364\/OE.16.021434","article-title":"Remote plethysmographic imaging using ambient light","volume":"16","author":"Verkruysse","year":"2008","journal-title":"Opt. Express"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"10762","DOI":"10.1364\/OE.18.010762","article-title":"Non-contact, automated cardiac pulse measurements using video imaging and blind source separation","volume":"18","author":"Poh","year":"2010","journal-title":"Opt. Express"},{"key":"ref_24","unstructured":"Lewandowska, M., Rumi\u0144ski, J., Kocejko, T., and Nowak, J. (2011, January 18\u201321). Measuring pulse rate with a webcam\u2014A non-contact method for evaluating cardiac activity. Proceedings of the 2011 Federated Conference on Computer Science and Information Systems (FedCSIS), Szczecin, Poland."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2878","DOI":"10.1109\/TBME.2013.2266196","article-title":"Robust Pulse Rate From Chrominance-Based rPPG","volume":"60","author":"Jeanne","year":"2013","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1913","DOI":"10.1088\/0967-3334\/35\/9\/1913","article-title":"Improved motion robustness of remote-PPG by using the blood volume pulse signature","volume":"35","year":"2014","journal-title":"Physiol. Meas."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1479","DOI":"10.1109\/TBME.2016.2609282","article-title":"Algorithmic Principles of Remote PPG","volume":"64","author":"Wang","year":"2017","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Pilz, C.S., Zaunseder, S., Krajewski, J., and Blazek, V. (2018, January 18\u201323). Local Group Invariance for Heart Rate Estimation from Face Videos in the Wild. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPRW.2018.00172"},{"key":"ref_29","unstructured":"Casado, C.A., and L\u00f3pez, M.B. (2022). Face2PPG: An unsupervised pipeline for blood volume pulse extraction from faces. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1186\/s12938-022-01037-z","article-title":"Performance analysis of remote photoplethysmography deep filtering using long short-term memory neural network","volume":"21","author":"Benezeth","year":"2022","journal-title":"Biomed. Eng. Online"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"2301","DOI":"10.1016\/S0735-1097(23)02745-6","article-title":"Camera-based remote photoplethysmography to measure heart rate and blood pressure in ambulatory patients with cardiovascular disease: Preliminary analysis","volume":"81","author":"Curran","year":"2023","journal-title":"J. Am. Coll. Cardiol."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Pourbemany, J., Essa, A., and Zhu, Y. (2021, January 16\u201319). Real-Time Video-based Heart and Respiration Rate Monitoring. Proceedings of the NAECON 2021\u2014IEEE National Aerospace and Electronics Conference, Dayton, OH, USA.","DOI":"10.1109\/NAECON49338.2021.9696378"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Negishi, T., Abe, S., Matsui, T., Liu, H., Kurosawa, M., Kirimoto, T., and Sun, G. (2020). Contactless Vital Signs Measurement System Using RGB-Thermal Image Sensors and Its Clinical Screening Test on Patients with Seasonal Influenza. Sensors, 20.","DOI":"10.3390\/s20082171"},{"key":"ref_34","unstructured":"Nagel, M., Fournarakis, M., Amjad, R.A., Bondarenko, Y., van Baalen, M., and Blankevoort, T. (2021). A White Paper on Neural Network Quantization. arXiv."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"109167","DOI":"10.1109\/ACCESS.2022.3214214","article-title":"Anatomy of Deep Learning Image Classification and Object Detection on Commercial Edge Devices: A Case Study on Face Mask Detection","volume":"10","author":"Kolosov","year":"2022","journal-title":"IEEE Access"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Kolosov, D., and Mporas, I. (2021, January 11\u201315). Face Masks Usage Monitoring for Public Health Security using Computer Vision on Hardware. Proceedings of the 2021 International Carnahan Conference on Security Technology (ICCST), Hatfield, UK.","DOI":"10.1109\/ICCST49569.2021.9717402"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Lin, J.-W., Lu, M.-H., and Lin, Y.-H. (2019, January 6\u20139). A Contactless Healthcare System with Face Recognition. Proceedings of the 2019 4th International Conference on Intelligent Green Building and Smart Grid (IGBSG), Yichang, China.","DOI":"10.1109\/IGBSG.2019.8886237"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Huang, H.-W., Rupp, P., Chen, J., Kemkar, A., Khandelwal, N., Ballinger, I., Chai, P., and Traverso, G. (November, January 30). Cost-Effective Solution of Remote Photoplethysmography Capable of Real-Time, Multi-Subject Monitoring with Social Distancing. Proceedings of the 2022 IEEE Sensors, Dallas, TX, USA.","DOI":"10.1109\/SENSORS52175.2022.9967120"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Lin, Y.-C., Chou, N.-K., Lin, G.-Y., Li, M.-H., and Lin, Y.-H. (2017). A Real-Time Contactless Pulse Rate and Motion Status Monitoring System Based on Complexion Tracking. Sensors, 17.","DOI":"10.3390\/s17071490"},{"key":"ref_40","unstructured":"David, R., Duke, J., Jain, A., Reddi, V.J., Jeffries, N., Li, J., Kreeger, N., Nappier, I., Natraj, M., and Wang, T. (2021). TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems. arXiv, 800\u2013811."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Shafi, O., Rai, C., Sen, R., and Ananthanarayanan, G. (2021, January 7\u20139). Demystifying TensorRT: Characterizing Neural Network Inference Engine on Nvidia Edge Devices. Proceedings of the 2021 IEEE International Symposium on Workload Characterization (IISWC), Storrs, CT, USA.","DOI":"10.1109\/IISWC53511.2021.00030"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Retsinas, G., Elafrou, A., Goumas, G., and Maragos, P. (2020). Weight Pruning via Adaptive Sparsity Los. arXiv.","DOI":"10.1109\/ICIP42928.2021.9506301"},{"key":"ref_43","unstructured":"Kokhazadeh, M., Keramidas, G., Kelefouras, V., and Stamoulis, I. (2022). Embedded Computer Systems: Architectures, Modeling, and Simulation, Proceedings of the 22nd International Conference, SAMOS 2022, Samos, Greece, 3\u20137 July 2022, Springer International Publishing."},{"key":"ref_44","unstructured":"Lugaresi, C., Tang, J., Nash, H., McClanahan, C., Uboweja, E., Hays, M., Zhang, F., Chang, C.-L., Yong, M.G., and Lee, J. (2019). MediaPipe: A Framework for Building Perception Pipelines. arXiv."},{"key":"ref_45","unstructured":"Bazarevsky, V., Kartynnik, Y., Vakunov, A., Raveendran, K., and Grundmann, M. (2019). BlazeFace: Sub-millisecond Neural Face Detection on Mobile GPUs. arXiv."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.Y., and Berg, A.C. (2016, January 11\u201314). SSD: Single shot multibox detector. Proceedings of the European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46448-0_2"},{"key":"ref_47","unstructured":"Kartynnik, Y., Ablavatski, A., Grishchenko, I., and Grundmann, M. (2019). Real-time Facial Surface Geometry from Monocular Video on Mobile GPUs. arXiv."},{"key":"ref_48","unstructured":"(2023, April 16). Getting Image Orientation and Bounding Box Coordinates\u2014Amazon Rekognition. Available online: https:\/\/docs.aws.amazon.com\/rekognition\/latest\/dg\/images-orientation.html."},{"key":"ref_49","unstructured":"(2023, April 24). Face mesh. Available online: https:\/\/developers.google.com\/static\/ar\/images\/augmented-faces\/augmented-faces-468-point-face-mesh.png."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/JTEHM.2018.2818687","article-title":"Algorithms for Monitoring Heart Rate and Respiratory Rate From the Video of a User\u2019s Face","volume":"6","author":"Sanyal","year":"2018","journal-title":"IEEE J. Transl. Eng. Health Med."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Lin, Y.-C., and Lin, Y.-H. (2017, January 11\u201315). A study of color illumination effect on the SNR of rPPG signals. Proceedings of the 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Jeju, Republic of Korea.","DOI":"10.1109\/EMBC.2017.8037807"},{"key":"ref_52","unstructured":"(2023, March 13). Raspberry Pi 4 Model B Specifications. Available online: https:\/\/www.raspberrypi.org\/products\/raspberry-pi-4-model-b\/."},{"key":"ref_53","unstructured":"(2023, March 13). Jetson Nano Developer Kit. Available online: https:\/\/developer.nvidia.com\/blog\/jetson-nano-ai-computing\/."},{"key":"ref_54","unstructured":"(2023, March 13). Jetson Xavier NX Series. Available online: https:\/\/www.nvidia.com\/en-us\/autonomous-machines\/embedded-systems\/jetson-xavier-nx\/."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/9\/4550\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:30:55Z","timestamp":1760124655000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/9\/4550"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,7]]},"references-count":54,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2023,5]]}},"alternative-id":["s23094550"],"URL":"https:\/\/doi.org\/10.3390\/s23094550","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,7]]}}}