{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,4]],"date-time":"2026-02-04T17:20:43Z","timestamp":1770225643509,"version":"3.49.0"},"reference-count":20,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2020,2,24]],"date-time":"2020-02-24T00:00:00Z","timestamp":1582502400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Monitoring breathing is important for a plethora of applications including, but not limited to, baby monitoring, sleep monitoring, and elderly care. This paper presents a way to fuse both vision-based and RF-based modalities for the task of estimating the breathing rate of a human. The modalities used are the F200 Intel\u00ae RealSenseTM RGB and depth (RGBD) sensor, and an ultra-wideband (UWB) radar. RGB image-based features and their corresponding image coordinates are detected on the human body and are tracked using the famous optical flow algorithm of Lucas and Kanade. The depth at these coordinates is also tracked. The synced-radar received signal is processed to extract the breathing pattern. All of these signals are then passed to a harmonic signal detector which is based on a generalized likelihood ratio test. Finally, a spectral estimation algorithm based on the reformed Pisarenko algorithm tracks the breathing fundamental frequencies in real-time, which are then fused into a one optimal breathing rate in a maximum likelihood fashion. We tested this multimodal set-up on 14 human subjects and we report a maximum error of     0.5     BPM compared to the true breathing rate.<\/jats:p>","DOI":"10.3390\/s20041229","type":"journal-article","created":{"date-parts":[[2020,2,25]],"date-time":"2020-02-25T04:21:26Z","timestamp":1582604486000},"page":"1229","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Multi-Modal, Remote Breathing Monitor"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8927-1314","authenticated-orcid":false,"given":"Nir","family":"Regev","sequence":"first","affiliation":[{"name":"School of Electrical and Computer Engineering, Ben-Gurion University of the Negev, Beer-Sheva 8410501, Israel"}]},{"given":"Dov","family":"Wulich","sequence":"additional","affiliation":[{"name":"School of Electrical and Computer Engineering, Ben-Gurion University of the Negev, Beer-Sheva 8410501, Israel"}]}],"member":"1968","published-online":{"date-parts":[[2020,2,24]]},"reference":[{"key":"ref_1","unstructured":"Kwon, S., Kim, H., and Park, K.S. (September, January 28). Validation of heart rate extraction using video imaging on a built-in camera system of a smartphone. Proceedings of the 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Diego, CA, USA."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Li, X., Chen, J., Zhao, G., and Pietik\u00e4inen, M. (2014, January 23\u201328). Remote Heart Rate Measurement from Face Videos under Realistic Situations. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.543"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"10762","DOI":"10.1364\/OE.18.010762","article-title":"Non-contact, automated cardiac pulse measurements using video imaging and blind source separation","volume":"18","author":"Poh","year":"2010","journal-title":"Opt. Express"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1109\/TBME.2010.2086456","article-title":"Advancements in Noncontact, Multiparameter Physiological Measurements Using a Webcam","volume":"58","author":"Poh","year":"2011","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Balakrishnan, G., Durand, F., and Guttag, J. (2013, January 23\u201328). Detecting Pulse from Head Motions in Video. Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition, CVPR \u201913, Portland, OR, USA.","DOI":"10.1109\/CVPR.2013.440"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2185520.2185561","article-title":"Eulerian Video Magnification for Revealing Subtle Changes in the World","volume":"31","author":"Wu","year":"2012","journal-title":"ACM Trans. Graph."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Chen, J., Chang, Z., Qiu, Q., Li, X., Sapiro, G., Bronstein, A., and Pietik\u00e4inen, M. (2016, January 12\u201315). RealSense= Real Heart Rate: Illumination Invariant Heart Rate Estimation from Videos. Proceedings of the 2016 Sixth International Conference on Image Processing Theory, Tools and Applications (IPTA), Oulu, Finland.","DOI":"10.1109\/IPTA.2016.7820970"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"N21","DOI":"10.1088\/0967-3334\/22\/3\/401","article-title":"Development of real-time image sequence analysis for evaluating posture change and respiratory rate of a subject in bed","volume":"22","author":"Nakajima","year":"2001","journal-title":"Physiol. Meas."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Regev, N., and Wulich, D. (2017, January 11\u201315). A simple, remote, video based breathing monitor. Proceedings of the 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Seogwipo, Korea.","DOI":"10.1109\/EMBC.2017.8037191"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Koo, Y.S., Ren, L., Wang, Y., and Fathy, A.E. (2013, January 2\u20137). UWB MicroDoppler Radar for human Gait analysis, tracking more than one person, and vital sign detection of moving persons. Proceedings of the 2013 IEEE MTT-S International Microwave Symposium Digest (IMS), Seattle, WA, USA.","DOI":"10.1109\/MWSYM.2013.6697702"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Tariq, A., and Shiraz, H.G. (2010, January 8\u20139). Doppler radar vital signs monitoring using wavelet transform. Proceedings of the 2010 Loughborough, Antennas and Propagation Conference (LAPC), Loughborough, UK.","DOI":"10.1109\/LAPC.2010.5666002"},{"key":"ref_12","unstructured":"Hsieh, C.H., Shen, Y.H., Chiu, Y.F., Chu, T.S., and Huang, Y.H. (2013, January 19\u201323). Human respiratory feature extraction on an UWB radar signal processing platform. Proceedings of the 2013 IEEE International Symposium on Circuits and Systems (ISCAS), Beijing, China."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"265","DOI":"10.2528\/PIER09120302","article-title":"Analysis of vital signs monitoring using an IR-UWB radar","volume":"100","author":"Lazaro","year":"2010","journal-title":"Prog. Electromagn. Res."},{"key":"ref_14","unstructured":"Shi, J., and Tomasi, C. (1994, January 21\u201323). Good Features to Track. Proceedings of the 1994 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA."},{"key":"ref_15","unstructured":"Lucas, B.D., and Kanade, T. (1981). An Iterative Image Registration Technique with an Application to Stereo Vision. Proceedings of the 7th International Joint Conference on Artificial Intelligence-Volume 2, Morgan Kaufmann Publishers Inc.. IJCAI\u201981."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1128","DOI":"10.1109\/TSP.2004.823473","article-title":"Reformulation of Pisarenko harmonic decomposition method for single-tone frequency estimation","volume":"52","author":"So","year":"2004","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_17","unstructured":"Haykin, S. (1996). Adaptive Filter Theory, Prentice-Hall, Inc.. [3rd ed.]."},{"key":"ref_18","unstructured":"Kay, S.M. (1993). Fundamentals of Statistical Signal Processing, Volume 2: Detection Theory, Prentice-Hall Inc."},{"key":"ref_19","unstructured":"Kay, S.M. (1993). Fundamentals of Statistical Signal Processing: Estimation Theory, Prentice-Hall, Inc."},{"key":"ref_20","unstructured":"(2019, December 13). Neulog\u2019s Respiration Monitor Belt logger NUL-236. Available online: https:\/\/neulog.com\/respiration-monitor-belt\/."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/4\/1229\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:01:12Z","timestamp":1760173272000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/4\/1229"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,2,24]]},"references-count":20,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2020,2]]}},"alternative-id":["s20041229"],"URL":"https:\/\/doi.org\/10.3390\/s20041229","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,2,24]]}}}