{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,31]],"date-time":"2025-10-31T13:37:12Z","timestamp":1761917832599,"version":"build-2065373602"},"reference-count":42,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2016,6,28]],"date-time":"2016-06-28T00:00:00Z","timestamp":1467072000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper is devoted to a new method of using Microsoft (MS) Kinect sensors for non-contact monitoring of breathing and heart rate estimation to detect possible medical and neurological disorders. Video sequences of facial features and thorax movements are recorded by MS Kinect image, depth and infrared sensors to enable their time analysis in selected regions of interest. The proposed methodology includes the use of computational methods and functional transforms for data selection, as well as their denoising, spectral analysis and visualization, in order to determine specific biomedical features. The results that were obtained verify the correspondence between the evaluation of the breathing frequency that was obtained from the image and infrared data of the mouth area and from the thorax movement that was recorded by the depth sensor. Spectral analysis of the time evolution of the mouth area video frames was also used for heart rate estimation. Results estimated from the image and infrared data of the mouth area were compared with those obtained by contact measurements by Garmin sensors (www.garmin.com). The study proves that simple image and depth sensors can be used to efficiently record biomedical multidimensional data with sufficient accuracy to detect selected biomedical features using specific methods of computational intelligence. The achieved accuracy for non-contact detection of breathing rate was 0.26% and the accuracy of heart rate estimation was 1.47% for the infrared sensor. The following results show how video frames with depth data can be used to differentiate different kinds of breathing. The proposed method enables us to obtain and analyse data for diagnostic purposes in the home environment or during physical activities, enabling efficient human\u2013machine interaction.<\/jats:p>","DOI":"10.3390\/s16070996","type":"journal-article","created":{"date-parts":[[2016,6,29]],"date-time":"2016-06-29T06:20:33Z","timestamp":1467181233000},"page":"996","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":64,"title":["Microsoft Kinect Visual and Depth Sensors for Breathing and Heart Rate Analysis"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0270-1738","authenticated-orcid":false,"given":"Ale\u0161","family":"Proch\u00e1zka","sequence":"first","affiliation":[{"name":"Department of Computing and Control Engineering, University of Chemistry and Technology, 166 28 Prague 6, Czech Republic"}]},{"given":"Martin","family":"Sch\u00e4tz","sequence":"additional","affiliation":[{"name":"Department of Computing and Control Engineering, University of Chemistry and Technology, 166 28 Prague 6, Czech Republic"}]},{"given":"Old\u0159ich","family":"Vy\u0161ata","sequence":"additional","affiliation":[{"name":"Department of Computing and Control Engineering, University of Chemistry and Technology, 166 28 Prague 6, Czech Republic"},{"name":"Czech Institute of Informatics, Robotics and Cybernetics, Czech Technical University, 166 36 Prague 6, Czech Republic"},{"name":"Department of Neurology, Faculty of Medicine in Hradec Kr\u00e1lov\u00e9, Charles University, 500 05 Hradec Kr\u00e1lov\u00e9, Czech Republic"}]},{"given":"Martin","family":"Vali\u0161","sequence":"additional","affiliation":[{"name":"Department of Neurology, Faculty of Medicine in Hradec Kr\u00e1lov\u00e9, Charles University, 500 05 Hradec Kr\u00e1lov\u00e9, Czech Republic"}]}],"member":"1968","published-online":{"date-parts":[[2016,6,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1049\/htl.2014.0063","article-title":"Respiratory rate detection algorithm based on RGB-D camera: Theoretical background and experimental results","volume":"1","author":"Benetazzo","year":"2014","journal-title":"Healthc. Technol. Lett."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"11204","DOI":"10.3390\/s140611204","article-title":"A Medical Cloud-Based Platform for Respiration Rate Measurement and Hierarchical Classification of Breath Disorders","volume":"14","author":"Fekr","year":"2014","journal-title":"Sensors"},{"key":"ref_3","first-page":"1","article-title":"Real-time stress assessment using thermal imaging","volume":"2015","author":"Hong","year":"2015","journal-title":"Vis. Comput."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2163","DOI":"10.4304\/jcp.8.9.2163-2166","article-title":"A Novel Thermal Measurement for Heart Rate","volume":"8","author":"Jing","year":"2013","journal-title":"J. Comput."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2543","DOI":"10.1109\/TMTT.2009.2029716","article-title":"Assessment of Heart Rate Variability and Respiratory Sinus Arrhythmia via Doppler Radar","volume":"57","author":"Massagram","year":"2009","journal-title":"IEEE Trans. Microw. Theory Technol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1153","DOI":"10.1109\/JBHI.2013.2291900","article-title":"A machine learning approach to improve contactless heart rate monitoring using a webcam","volume":"18","author":"Monkaresi","year":"2014","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2760","DOI":"10.1109\/TBME.2014.2327024","article-title":"Noncontact monitoring breathing pattern, exhalation flow rate and pulse transit time","volume":"61","author":"Shao","year":"2014","journal-title":"IEEE Trans. Bio-Med. Eng."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"23004","DOI":"10.3390\/s150923004","article-title":"New fast fall detection method based on spatio-temporal context tracking of head by using depth images","volume":"15","author":"Yang","year":"2015","journal-title":"Sensors"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Balakrishnan, G., Durand, F., and Gutag, J. (2013, January 20\u201325). Detecting pulse from head motions in video. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR, Portland, OR, USA.","DOI":"10.1109\/CVPR.2013.440"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"10762","DOI":"10.1364\/OE.18.010762","article-title":"Non-contact, automated cardiac pulse measurements using video imaging and blind source separation","volume":"18","author":"Poh","year":"2010","journal-title":"Opt. Express"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"24297","DOI":"10.3390\/s150924297","article-title":"Leveraging Two Kinect Sensors for Accurate Full-Body Motion Capture","volume":"15","author":"Gao","year":"2015","journal-title":"Sensors"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"13070","DOI":"10.3390\/rs71013070","article-title":"Assessment and Calibration of a RGB-D Camera (Kinect v2 Sensor) Towards a Potential Use for Close-Range 3D Modeling","volume":"7","author":"Lachat","year":"2015","journal-title":"Remote Sens."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1016\/j.dsp.2015.05.011","article-title":"Bayesian Classification and Analysis of Gait Disorders Using Image and Depth Sensors of Microsoft Kinect","volume":"47","author":"Schatz","year":"2015","journal-title":"Digit. Signal Process."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1621","DOI":"10.1007\/s00521-015-1827-x","article-title":"Use of Image and Depth Sensors of the Microsoft Kinect for the Detection of Gait Disorders","volume":"26","author":"Schatz","year":"2015","journal-title":"Neural Comput. Appl."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Yang, C., Cheung, G., and Stankovic, V. (July, January 29). Estimating heart rate via depth video motion tracking. Proceedings of the IEEE International Conference on Multimedia and Expo, ICME-2015, Torino, Italy.","DOI":"10.1109\/ICME.2015.7177517"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1007\/s00371-013-0819-z","article-title":"Content-based 3D model retrieval using a single depth image from a low-cost 3D camera","volume":"29","author":"Bae","year":"2013","journal-title":"Vis. Comput."},{"key":"ref_17","first-page":"1","article-title":"Motion tracking and gait feature estimation for recognising Parkinson\u2019s disease using MS Kinect","volume":"14","author":"Schatz","year":"2015","journal-title":"BioMed. Eng. OnLine"},{"key":"ref_18","unstructured":"Yu, M., Wu, H., Liou, J., Lee, M., and Hung, Y. (2012, January 1\u20134). Multiparameter sleep monitoring using a depth camera. Proceedings of the BIOSTEC 2012, Vilamoura, Portugal."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1133","DOI":"10.1007\/s00371-013-0894-1","article-title":"Real-time and robust hand tracking with a single depth camera","volume":"30","author":"Ma","year":"2014","journal-title":"Vis. Comput."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"A121","DOI":"10.1136\/thoraxjnl-2011-201054c.134","article-title":"P134 Validating structured light plethysmography (SLP) as a non-invasive method of measuring lung function when compared to Spirometry","volume":"66","author":"Alimohamed","year":"2011","journal-title":"Thorax"},{"key":"ref_21","first-page":"A2528","article-title":"Tidal Breathing Parameters Measurement by Structured Light Plethysmography (SLP) and Spirometry","volume":"B18","author":"Brand","year":"2010","journal-title":"Am. J. Resp. Crit. Care"},{"key":"ref_22","unstructured":"Loblaw, A., Nielsen, J., Okoniewski, M., and Lakhani, M.A. (2013, January 25\u201328). Remote respiratory sensing with an infrared camera using the Kinect infrared projector. Proceedings of the 2013 World Congress in Computer Science, Computer Engineering and Alied Computing, WORLDCOMP, Las Vegas, NV, USA."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1109\/MEMB.2006.1636352","article-title":"Noncontact measurement of breathing function","volume":"25","author":"Murthy","year":"2014","journal-title":"IEEE Eng. Med. Biol. Mag."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"396","DOI":"10.1109\/TBME.2013.2280132","article-title":"Unconstrained video monitoring of breathing behavior and application to diagnosis of sleep apnea","volume":"61","author":"Wang","year":"2014","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_25","unstructured":"Falie, D., Ichim, M., and David, L. (2008, January 7\u20139). Respiratory motion visualization and the sleep apnea diagnosis with the time of flight (ToF) camera. Proceedings of the 1st WSEAS International Conference on VISUALIZATION, IMAGING and SIMULATION, VIS\u201908, Bucharest, Romania."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"6383","DOI":"10.3390\/s150306383","article-title":"Assessment of Human Respiration Patterns via Noncontact Sensing Using Doppler Multi-Radar System","volume":"15","author":"Gu","year":"2015","journal-title":"Sensors"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"15371","DOI":"10.3390\/s140815371","article-title":"An Ultrasonic Contactless Sensor for Breathing Monitoring","volume":"14","author":"Arlotto","year":"2014","journal-title":"Sensors"},{"key":"ref_28","first-page":"18950","article-title":"Monitoring of Weekly Sleep Pattern Variations at Home with a Contactless Biomotion Sensor","volume":"14","author":"Hashizaki","year":"2014","journal-title":"Sensors"},{"key":"ref_29","first-page":"11","article-title":"Force Sensitive Resistance Based Heart Beat Monitoring for Health Care System","volume":"4","author":"Pandiyan","year":"2014","journal-title":"Int. J. Inform. Sci. Technol."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Gargiulo, G.D., and McEwan, A. (2011). Applied Biomedical Engineering, INTECH.","DOI":"10.5772\/764"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1349","DOI":"10.1007\/s00521-014-1622-0","article-title":"Discrimination of Axonal Neuropathy Using Sensitivity and Specificity Statistical Measures","volume":"25","year":"2014","journal-title":"Neural Comput. Appl."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2016\/4351435","article-title":"A Human Activity Recognition System Using Skeleton Data from RGBD Sensors","volume":"2016","author":"Cippitelli","year":"2016","journal-title":"Comput. Intell. Neurosci."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1109\/MSP.2015.2489978","article-title":"Sensors in Assisted Living","volume":"33","author":"Erden","year":"2016","journal-title":"IEEE Signal Proc. Mag."},{"key":"ref_34","first-page":"1","article-title":"A depth camera motion analysis framework for tele-rehabilitation: Motion capture and person-centric kinematics analysis","volume":"2016","author":"Ye","year":"2016","journal-title":"IEEE J. Sel. Top. Signal Process."},{"key":"ref_35","unstructured":"Yang, C., Mao, Y., Cheung, G., Stankovic, V., and Chan, K. (2014, January 14\u201318). Monitoring via Depth Video Recording & Analysis. Proceedings of the 5th IEEE International Workshop on Hot Topics in 3D, Hot3D, Chengdu, China."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Yang, C., Mao, Y., Cheung, G., Stankovic, V., and Chan, K. (2014, January 7\u20139). Graph-Based Depth Video Denoising and Event Detection for Sleep Monitoring. Proceedings of the IEEE International Workshop on Multimedia Signal Processing, Jakarta, Indonesia.","DOI":"10.1109\/MMSP.2014.6958802"},{"key":"ref_37","first-page":"43","article-title":"Lip Contour Detection Techniques Based on Front View Of Face","volume":"2","author":"Bandyopadhyay","year":"2011","journal-title":"J. Glob. Res. Comput. Sci."},{"key":"ref_38","first-page":"51","article-title":"Efficient Eyes and Mouth Detection Algorithm Using Combination of Viola Jones and Skin Color Pixel Detection","volume":"3","author":"Khan","year":"2013","journal-title":"Int. J. Eng. Appl. Sci."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1437","DOI":"10.3390\/s120201437","article-title":"Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications","volume":"12","author":"Khoshelham","year":"2012","journal-title":"Sensors"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"222","DOI":"10.1007\/s10851-011-0274-2","article-title":"Image Sequence Interpolation Using Optimal Control","volume":"41","author":"Chen","year":"2011","journal-title":"J. Math. Imaging Vis."},{"key":"ref_41","first-page":"1","article-title":"Frame Interpolation Based on Visual Correspondence and Coherency Sensitive Hashing","volume":"13","author":"Zi","year":"2013","journal-title":"Math. Probl. Eng."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Merget, D., Eckl, T., Schw\u00f6rer, M., Tiefenbacher, P., and Rigoll, G. (2016, January 7\u20139). Capturing Facial Videos with Kinect 2.0: A Multithreaded Open Source Tool and Database. Proceedings of the IEEE Winter Conference on Applications of Computer Vision, WACV 2016, Lake Placid, NY, USA.","DOI":"10.1109\/WACV.2016.7477559"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/16\/7\/996\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T19:24:58Z","timestamp":1760210698000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/16\/7\/996"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,6,28]]},"references-count":42,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2016,7]]}},"alternative-id":["s16070996"],"URL":"https:\/\/doi.org\/10.3390\/s16070996","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2016,6,28]]}}}