{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,7]],"date-time":"2025-11-07T09:22:36Z","timestamp":1762507356474,"version":"build-2065373602"},"reference-count":50,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2018,3,20]],"date-time":"2018-03-20T00:00:00Z","timestamp":1521504000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Monitoring of cardiopulmonary activity is a challenge when attempted under adverse conditions, including different sleeping postures, environmental settings, and an unclear region of interest (ROI). This study proposes an efficient remote imaging system based on a Microsoft Kinect v2 sensor for the observation of cardiopulmonary-signal-and-detection-related abnormal cardiopulmonary events (e.g., tachycardia, bradycardia, tachypnea, bradypnea, and central apnoea) in many possible sleeping postures within varying environmental settings including in total darkness and whether the subject is covered by a blanket or not. The proposed system extracts the signal from the abdominal-thoracic region where cardiopulmonary activity is most pronounced, using a real-time image sequence captured by Kinect v2 sensor. The proposed system shows promising results in any sleep posture, regardless of illumination conditions and unclear ROI even in the presence of a blanket, whilst being reliable, safe, and cost-effective.<\/jats:p>","DOI":"10.3390\/s18030920","type":"journal-article","created":{"date-parts":[[2018,3,20]],"date-time":"2018-03-20T06:57:11Z","timestamp":1521529031000},"page":"920","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Detection of Cardiopulmonary Activity and Related Abnormal Events Using Microsoft Kinect Sensor"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8840-9235","authenticated-orcid":false,"given":"Ali","family":"Al-Naji","sequence":"first","affiliation":[{"name":"School of Engineering, University of South Australia, Mawson Lakes, SA 5095, Australia"},{"name":"Electrical Engineering Technical College, Middle Technical University, Al Doura 10022, Baghdad, Iraq"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6496-0543","authenticated-orcid":false,"given":"Javaan","family":"Chahl","sequence":"additional","affiliation":[{"name":"School of Engineering, University of South Australia, Mawson Lakes, SA 5095, Australia"},{"name":"Joint and Operations Analysis Division, Defence Science and Technology Group, Melbourne, VIC 3207, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,3,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1364\/AOP.3.000128","article-title":"Structured-light 3D surface imaging: A tutorial","volume":"3","author":"Geng","year":"2011","journal-title":"Adv. Opt. Photonics"},{"key":"ref_2","unstructured":"Mutto, C.D., Zanuttigh, P., and Cortelazzo, G.M. (2012). Time-of-Flight Cameras and Microsoft Kinect (TM), Springer Publishing Company Incorporated."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"4275","DOI":"10.1109\/JSEN.2015.2416651","article-title":"Evaluating and improving the depth accuracy of Kinect for Windows v2","volume":"15","author":"Yang","year":"2015","journal-title":"Sens. J. IEEE"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.cviu.2015.05.006","article-title":"Kinect range sensing: Structured-light versus Time-of-Flight Kinect","volume":"139","author":"Sarbolandi","year":"2015","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Samir, M., Golkar, E., and Rahni, A.A.A. (2015, January 19\u201321). Comparison between the KinectTM V1 and KinectTM V2 for respiratory motion tracking. Proceedings of the 2015 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), Kuala Lumpur, Malaysia.","DOI":"10.1109\/ICSIPA.2015.7412180"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Kim, C., Yun, S., Jung, S.-W., and Won, C.S. (2015). Color and depth image correspondence for Kinect v2. Advanced Multimedia and Ubiquitous Engineering, Springer.","DOI":"10.1007\/978-3-662-47487-7_17"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1007\/978-3-319-54427-4_3","article-title":"Comparison of kinect v1 and v2 depth images in terms of accuracy and precision","volume":"Volume 10117","author":"Stricker","year":"2017","journal-title":"Computer Vision\u2013ACCV 2016 Workshops: ACCV 2016"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2682","DOI":"10.1118\/1.4704644","article-title":"A real-time respiratory motion monitoring system using KINECT: Proof of concept","volume":"39","author":"Xia","year":"2012","journal-title":"Med. Phys."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Smilkstein, T., Buenrostro, M., Kenyon, A., Lienemann, M., and Larson, G. (2014, January 8\u201310). Heart rate monitoring using Kinect and color amplification. Proceedings of the Healthcare Innovation Conference (HIC), Seattle, WA, USA.","DOI":"10.1109\/HIC.2014.7038874"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1145\/2185520.2185561","article-title":"Eulerian video magnification for revealing subtle changes in the world","volume":"31","author":"Wu","year":"2012","journal-title":"ACM Trans. Graph."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Bernacchia, N., Scalise, L., Casacanditella, L., Ercoli, I., Marchionni, P., and Tomasini, E.P. (2014, January 11\u201312). Non contact measurement of heart and respiration rates based on Kinect\u2122. Proceedings of the 2014 IEEE International Symposium on Medical Measurements and Applications (MeMeA), Lisboa, Portugal.","DOI":"10.1109\/MeMeA.2014.6860065"},{"key":"ref_12","unstructured":"Yu, M.-C., Wu, H., Liou, J.-L., Lee, M.-S., and Hung, Y.-P. (2012, January 1\u20134). Breath and position monitoring during sleeping with a depth camera. Proceedings of the International Conference on Health Informatics (HEALTHINF-2012), Algarve, Portugal."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Yang, C., Cheung, G., Chan, K.L., and Stankovic, V. (2014, January 14\u201318). Sleep monitoring via depth video compression & analysis. Proceedings of the 2014 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), Chengdu, China.","DOI":"10.1109\/ICMEW.2014.6890645"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Tanaka, M. (June, January 31). Application of depth sensor for breathing rate counting. Proceedings of the 2015 10th Asian Control Conference (ASCC), Kota Kinabalu, Malaysia.","DOI":"10.1109\/ASCC.2015.7244556"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"270","DOI":"10.1515\/cdbme-2015-0067","article-title":"Respiratory surface motion measurement by Microsoft Kinect","volume":"1","author":"Gauer","year":"2015","journal-title":"Curr. Dir. Biomed. Eng."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Aoki, H., Nakamura, H., Fumoto, K., Nakahara, K., and Teraoka, M. (2015, January 11\u201313). Basic study on non-contact respiration measurement during exercise tolerance test by using kinect sensor. Proceedings of the 2015 IEEE\/SICE International Symposium on System Integration (SII), Nagoya, Japan.","DOI":"10.1109\/SII.2015.7404981"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Centonze, F., Schatz, M., Prochazka, A., Kuchynka, J., Vysata, O., Cejnar, P., and Valis, M. (2015, January 29\u201330). Feature extraction using MS Kinect and data fusion in analysis of sleep disorders. Proceedings of the 2015 International Workshop on Computational Intelligence for Multimedia Understanding (IWCIM), Prague, Czech Republic.","DOI":"10.1109\/IWCIM.2015.7347069"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Yang, C., Cheung, G., and Stankovic, V. (July, January 29). Estimating heart rate via depth video motion tracking. Proceedings of the 2015 IEEE International Conference on Multimedia and Expo (ICME), Turin, Italy.","DOI":"10.1109\/ICME.2015.7177517"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1631","DOI":"10.1007\/s11517-015-1433-1","article-title":"Chest wall motion analysis in healthy volunteers and adults with cystic fibrosis using a novel Kinect-based motion tracking system","volume":"54","author":"Harte","year":"2016","journal-title":"Med. Boil. Eng. Comput."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Tahavori, F., Adams, E., Dabbs, M., Aldridge, L., Liversidge, N., Donovan, E., Jordan, T., Evans, P., and Wells, K. (2015, January 21). Combining marker-less patient setup and respiratory motion monitoring using low cost 3D camera technology. Proceedings of the SPIE Medical Imaging. International Society for Optics and Photonics, Orlando, FL, USA.","DOI":"10.1117\/12.2082726"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"20","DOI":"10.4236\/ojmi.2016.61003","article-title":"Markerless Respiratory Motion Tracking Using Single Depth Camera","volume":"6","author":"Kumagai","year":"2016","journal-title":"Open J. Med. Imaging"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1155\/2015\/875371","article-title":"Sleep monitoring system using kinect sensor","volume":"2015","author":"Lee","year":"2015","journal-title":"Int. J. Distrib. Sens. Netw."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Gambi, E., Agostinelli, A., Belli, A., Burattini, L., Cippitelli, E., Fioretti, S., Pierleoni, P., Ricciuti, M., Sbrollini, A., and Spinsante, S. (2017). Heart Rate Detection Using Microsoft Kinect: Validation and Comparison to Wearable Devices. Sensors, 17.","DOI":"10.3390\/s17081776"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1625","DOI":"10.1109\/TMM.2017.2672198","article-title":"Estimating heart rate and rhythm via 3D motion tracking in depth video","volume":"19","author":"Yang","year":"2017","journal-title":"IEEE Trans. Multimed."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Al-Naji, A., Gibson, K., Lee, S.-H., and Chahl, J. (2017). Real time apnoea monitoring of children using the microsoft kinect sensor: A pilot study. Sensors, 17.","DOI":"10.3390\/s17020286"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1016\/j.bspc.2014.03.004","article-title":"Non-contact heart rate and heart rate variability measurements: A review","volume":"13","author":"Kranjec","year":"2014","journal-title":"Biomed. Signal Process. Control"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"15776","DOI":"10.1109\/ACCESS.2017.2735419","article-title":"Monitoring of cardiorespiratory signal: Principles of remote measurements and review of methods","volume":"5","author":"Alnaji","year":"2017","journal-title":"IEEE Access"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Papon, T.I., Ahmad, I., Saquib, N., and Rahman, A. (2015, January 5\u20137). Non-invasive heart rate measuring smartphone applications using on-board cameras: A short survey. Proceedings of the 2015 International Conference on Networking Systems and Security (NSysS), Dhaka, Bangladesh.","DOI":"10.1109\/NSysS.2015.7043533"},{"key":"ref_29","first-page":"721","article-title":"Signs of respiratory disease","volume":"85","author":"Alonso","year":"2006","journal-title":"Apoptosis"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"292","DOI":"10.1049\/iet-ipr.2016.0569","article-title":"Quality index evaluation of videos based on fuzzy interface system","volume":"11","author":"Alnaji","year":"2017","journal-title":"IET Image Proc."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1186\/s12938-017-0395-y","article-title":"Remote monitoring of cardiorespiratory signals from a hovering unmanned aerial vehicle","volume":"16","author":"Alnaji","year":"2017","journal-title":"Biomed. Eng. Online"},{"key":"ref_32","first-page":"1","article-title":"Non-contact heart activity measurement system based on video imaging analysis","volume":"31","author":"Ali","year":"2017","journal-title":"Int. J. Pattern Recognit. Artif. Intell."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1016\/j.bspc.2014.06.009","article-title":"Improved complete ensemble EMD: A suitable tool for biomedical signal processing","volume":"14","author":"Colominas","year":"2014","journal-title":"Biomed. Signal Proc. Control"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"903","DOI":"10.1098\/rspa.1998.0193","article-title":"The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis","volume":"454","author":"Huang","year":"1998","journal-title":"Proc. Math Phys. Eng. Sci."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1142\/S1793536909000047","article-title":"Ensemble empirical mode decomposition: A noise-assisted data analysis method","volume":"1","author":"Wu","year":"2009","journal-title":"Adv. Adapt. Data Anal."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Torres, M.E., Colominas, M.A., Schlotthauer, G., and Flandrin, P. (2011, January 22\u201327). A complete ensemble empirical mode decomposition with adaptive noise. Proceedings of the 2011 IEEE international conference on Acoustics, Speech and Signal Processing (ICASSP), Prague, Czech Republic.","DOI":"10.1109\/ICASSP.2011.5947265"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/JTEHM.2017.2757485","article-title":"Simultaneous tracking of cardiorespiratory signals for multiple persons using a machine vision system with noise artifact removal","volume":"5","author":"Alnaji","year":"2017","journal-title":"IEEE J. Transl. Eng. Health Med."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1505","DOI":"10.1109\/TNN.2007.894017","article-title":"Analysis and online realization of the CCA approach for blind source separation","volume":"18","author":"Liu","year":"2007","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"3918","DOI":"10.1109\/TSP.2009.2021636","article-title":"Joint blind source separation by multiset canonical correlation analysis","volume":"57","author":"Li","year":"2009","journal-title":"IEEE Trans. Signal Proc."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1186\/1687-6180-2012-127","article-title":"Removal of muscle artifact from EEG data: Comparison between stochastic (ICA and CCA) and deterministic (EMD and wavelet-based) approaches","volume":"2012","author":"Safieddine","year":"2012","journal-title":"EURASIP J. Adv. Signal Proc."},{"key":"ref_41","unstructured":"Lin, Y.-P., Wang, Y., and Jung, T.-P. (2013, January 3\u20137). A mobile SSVEP-based brain-computer interface for freely moving humans: The robustness of canonical correlation analysis to motion artifacts. Proceedings of the 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Osaka, Japan."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1177\/155005941004100111","article-title":"Online removal of muscle artifact from electroencephalogram signals based on canonical correlation analysis","volume":"41","author":"Gao","year":"2010","journal-title":"Clin. EEG Neurosci."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"2583","DOI":"10.1109\/TBME.2006.879459","article-title":"Canonical correlation analysis applied to remove muscle artifacts from the electroencephalogram","volume":"53","author":"Vergult","year":"2006","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"762","DOI":"10.1109\/JSEN.2015.2490038","article-title":"A Blind Source Separation Framework for Monitoring Heart Beat Rate Using Nanofiber-Based Strain Sensors","volume":"16","author":"Zou","year":"2016","journal-title":"IEEE Sens. J."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1109\/TBME.2012.2225427","article-title":"The use of ensemble empirical mode decomposition with canonical correlation analysis as a novel artifact removal technique","volume":"60","author":"Sweeney","year":"2013","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1986","DOI":"10.1109\/JSEN.2015.2506982","article-title":"Removing Muscle Artifacts From EEG Data: Multichannel or Single-Channel Techniques?","volume":"16","author":"Chen","year":"2016","journal-title":"IEEE Sens. J."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Mart\u00ednez-Mart\u00edn, E., and del Pobil, \u00c1.P. (2012). Motion Detection in Static Backgrounds. Robust Motion Detection in Real-Life Scenarios, Springer.","DOI":"10.1007\/978-1-4471-4216-4"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Zuiderveld, K. (1994). Contrast Limited Adaptive Histogram Equalization, Academic Press Professional, Inc.. Graphics Gems IV.","DOI":"10.1016\/B978-0-12-336156-1.50061-6"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"914","DOI":"10.1542\/peds.111.4.914","article-title":"Apnea, sudden infant death syndrome, and home monitoring","volume":"111","author":"Pediatrics","year":"2003","journal-title":"Pediatrics"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"931","DOI":"10.1016\/j.ijnurstu.2009.10.001","article-title":"Statistical methods for assessing agreement between two methods of clinical measurement","volume":"47","author":"Bland","year":"2010","journal-title":"Int. J. Nurs. Stud."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/3\/920\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T14:57:46Z","timestamp":1760194666000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/3\/920"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,3,20]]},"references-count":50,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2018,3]]}},"alternative-id":["s18030920"],"URL":"https:\/\/doi.org\/10.3390\/s18030920","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2018,3,20]]}}}