{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,8]],"date-time":"2026-02-08T05:59:44Z","timestamp":1770530384313,"version":"3.49.0"},"reference-count":34,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2021,2,21]],"date-time":"2021-02-21T00:00:00Z","timestamp":1613865600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"German Research Foundation (Deutsche Forschungsgemeinschaft)","award":["LE 817\/26-1"],"award-info":[{"award-number":["LE 817\/26-1"]}]},{"name":"German Research Foundation (Deutsche Forschungsgemeinschaft)","award":["LE 817\/32-1"],"award-info":[{"award-number":["LE 817\/32-1"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Infrared thermography for camera-based skin temperature measurement is increasingly used in medical practice, e.g., to detect fevers and infections, such as recently in the COVID-19 pandemic. This contactless method is a promising technology to continuously monitor the vital signs of patients in clinical environments. In this study, we investigated both skin temperature trend measurement and the extraction of respiration-related chest movements to determine the respiratory rate using low-cost hardware in combination with advanced algorithms. In addition, the frequency of medical examinations or visits to the patients was extracted. We implemented a deep learning-based algorithm for real-time vital sign extraction from thermography images. A clinical trial was conducted to record data from patients on an intensive care unit. The YOLOv4-Tiny object detector was applied to extract image regions containing vital signs (head and chest). The infrared frames were manually labeled for evaluation. Validation was performed on a hold-out test dataset of 6 patients and revealed good detector performance (0.75 intersection over union, 0.94 mean average precision). An optical flow algorithm was used to extract the respiratory rate from the chest region. The results show a mean absolute error of 2.69 bpm. We observed a computational performance of 47 fps on an NVIDIA Jetson Xavier NX module for YOLOv4-Tiny, which proves real-time capability on an embedded GPU system. In conclusion, the proposed method can perform real-time vital sign extraction on a low-cost system-on-module and may thus be a useful method for future contactless vital sign measurements.<\/jats:p>","DOI":"10.3390\/s21041495","type":"journal-article","created":{"date-parts":[[2021,2,21]],"date-time":"2021-02-21T22:04:15Z","timestamp":1613945055000},"page":"1495","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":53,"title":["A Deep Learning-Based Camera Approach for Vital Sign Monitoring Using Thermography Images for ICU Patients"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3842-6897","authenticated-orcid":false,"given":"Simon","family":"Lyra","sequence":"first","affiliation":[{"name":"Medical Information Technology, Helmholtz Institute for Biomedical Engineering, RWTH Aachen University, 52074 Aachen, Germany"}]},{"given":"Leon","family":"Mayer","sequence":"additional","affiliation":[{"name":"Medical Information Technology, Helmholtz Institute for Biomedical Engineering, RWTH Aachen University, 52074 Aachen, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3937-9778","authenticated-orcid":false,"given":"Liyang","family":"Ou","sequence":"additional","affiliation":[{"name":"Medical Information Technology, Helmholtz Institute for Biomedical Engineering, RWTH Aachen University, 52074 Aachen, Germany"}]},{"given":"David","family":"Chen","sequence":"additional","affiliation":[{"name":"Eastern Health Clinical School, Monash University Melbourne, Box Hill, VIC 3128, Australia"}]},{"given":"Paddy","family":"Timms","sequence":"additional","affiliation":[{"name":"Eastern Health Clinical School, Monash University Melbourne, Box Hill, VIC 3128, Australia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6595-3166","authenticated-orcid":false,"given":"Andrew","family":"Tay","sequence":"additional","affiliation":[{"name":"Eastern Health Clinical School, Monash University Melbourne, Box Hill, VIC 3128, Australia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4578-9394","authenticated-orcid":false,"given":"Peter Y.","family":"Chan","sequence":"additional","affiliation":[{"name":"Eastern Health Clinical School, Monash University Melbourne, Box Hill, VIC 3128, Australia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9512-2910","authenticated-orcid":false,"given":"Bergita","family":"Ganse","sequence":"additional","affiliation":[{"name":"Research Centre for Musculoskeletal Science and Sports Medicine, Manchester Metropolitan University, Manchester M1 5GD, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6898-6887","authenticated-orcid":false,"given":"Steffen","family":"Leonhardt","sequence":"additional","affiliation":[{"name":"Medical Information Technology, Helmholtz Institute for Biomedical Engineering, RWTH Aachen University, 52074 Aachen, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7948-8181","authenticated-orcid":false,"given":"Christoph","family":"Hoog Antink","sequence":"additional","affiliation":[{"name":"Medical Information Technology, Helmholtz Institute for Biomedical Engineering, RWTH Aachen University, 52074 Aachen, Germany"},{"name":"Biomedical Engineering, Electrical Engineering and Information Technology, TU Darmstadt, 64289 Darmstadt, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2021,2,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"e16261","DOI":"10.1097\/MD.0000000000016261","article-title":"Mortality rate and other clinical features observed in Open vs closed format intensive care units: A systematic review and meta-analysis","volume":"98","author":"Yang","year":"2019","journal-title":"Medicine"},{"key":"ref_2","unstructured":"Goldfain, A., Smith, B., Arabandi, S., Brochhausen, M., and Hogan, W.R. Vital Sign Ontology. Proceedings of the Workshop on Bio-Ontologies."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1097\/CCM.0b013e31822f061d","article-title":"Determinants of temperature abnormalities and influence on outcome of critical illness","volume":"40","author":"Laupland","year":"2011","journal-title":"Crit. Care Med."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1159\/000505126","article-title":"Admission Body Temperature in Critically Ill Patients as an Independent Risk Predictor for Overall Outcome","volume":"29","author":"Erkens","year":"2019","journal-title":"Med. Princ. Pract."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"657","DOI":"10.5694\/j.1326-5377.2008.tb01825.x","article-title":"Respiratory rate: The neglected vital sign","volume":"188","author":"Cretikos","year":"2008","journal-title":"Med. J. Aust."},{"key":"ref_6","unstructured":"OECD\/European Union (2016) (2018). Mortality from respiratory diseases. Health at a Glance: Europe 2018: State of Health in the EU Cycle, OECD Publishing."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1016\/j.aenj.2016.12.003","article-title":"Accurate respiratory rates count: So should you!","volume":"20","author":"Flenady","year":"2017","journal-title":"Australas. Emerg. Nurs. J. AENJ"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1111\/j.1552-6909.2001.tb01520.x","article-title":"Neonatal Skin Care: Clinical Outcomes of the AWHONN\/NANN Evidence-Based Clinical Practice Guideline","volume":"30","author":"Lund","year":"2001","journal-title":"J. Obstet. Gynecol. Neonatal Nurs."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1055\/s-0039-1677914","article-title":"A Broader Look: Camera-Based Vital Sign Estimation across the Spectrum","volume":"28","author":"Lyra","year":"2019","journal-title":"Yearb. Med. Inform."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1117\/12.407646","article-title":"Photoplethysmography imaging: A new noninvasive and noncontact method for mapping of the dermal perfusion changes","volume":"Volume 4163","author":"Wu","year":"2000","journal-title":"Proceedings of the Optical Techniques and Instrumentation for the Measurement of Blood Composition, Structure, and Dynamics"},{"key":"ref_11","unstructured":"Murthy, R., Pavlidis, I., and Tsiamyrtzis, P. (2004, January 1\u20135). Touchless monitoring of breathing function. Proceedings of the The 26th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Francisco, CA, USA."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1186\/1475-925X-10-93","article-title":"Neonatal non-contact respiratory monitoring based on real-time infrared thermography","volume":"10","author":"Abbas","year":"2011","journal-title":"Biomed. Eng. Online"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"877","DOI":"10.1111\/j.1469-8986.2010.01167.x","article-title":"A novel method for extracting respiration rate and relative tidal volume from infrared thermography","volume":"48","author":"Lewis","year":"2011","journal-title":"Psychophysiology"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"4378","DOI":"10.1364\/BOE.6.004378","article-title":"Remote monitoring of breathing dynamics using infrared thermography","volume":"6","author":"Pereira","year":"2015","journal-title":"Biomed. Opt. Express"},{"key":"ref_15","unstructured":"Marzec, M., and Koprowski, R. (2018). Noncontact Monitoring of Vital Signs with RGB and Infrared Camera and Its Application to Screening of Potential Infection. Non-Invasive Diagnostic Methods, IntechOpen. Chapter 4."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Elphick, H., Alkali, A., Kingshott, R., Burke, D., and Saatchi, R. (2015). Thermal imaging method for measurement of respiratory rate. Eur. Respir. J., 46.","DOI":"10.1183\/13993003.congress-2015.PA1260"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Schroff, F., Kalenichenko, D., and Philbin, J. (2015, January 7\u201312). FaceNet: A unified embedding for face recognition and clustering. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298682"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Blanc-Talon, J., Penne, R., Philips, W., Popescu, D., and Scheunders, P. (2017). Face Detection in Thermal Infrared Images: A Comparison of Algorithm- and Machine-Learning-Based Approaches. Advanced Concepts for Intelligent Vision Systems, Springer International Publishing.","DOI":"10.1007\/978-3-319-70353-4"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Kwasniewska, A., Ruminski, J., and Szankin, M. (2019). Improving Accuracy of Contactless Respiratory Rate Estimation by Enhancing Thermal Sequences with Deep Neural Networks. Appl. Sci., 9.","DOI":"10.3390\/app9204405"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1016\/j.infrared.2019.103117","article-title":"Non-contact monitoring of human respiration using infrared thermography and machine learning","volume":"104","author":"Jagadev","year":"2020","journal-title":"Infrared Phys. Technol."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"035007","DOI":"10.1088\/2057-1976\/ab7a54","article-title":"Human respiration monitoring using infrared thermography and artificial intelligence","volume":"6","author":"Jagadev","year":"2020","journal-title":"Biomed. Phys. Eng. Express"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Pereira, C., Dohmeier, H., Kunczik, J., Hochhausen, N., Tolba, R., and Czaplik, M. (2019). Contactless monitoring of heart and respiratory rate in anesthetized pigs using infrared thermography. PLoS ONE, 14.","DOI":"10.1371\/journal.pone.0224747"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"4848","DOI":"10.1364\/BOE.397188","article-title":"Multi-Camera Infrared Thermography for Infant Respiration Monitoring","volume":"11","author":"Lorato","year":"2020","journal-title":"Biomed. Opt. Express"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1275","DOI":"10.1007\/s10877-019-00437-2","article-title":"Estimation of respiratory rate using infrared video in an inpatient population: An observational study","volume":"34","author":"Chan","year":"2019","journal-title":"J. Clin. Monit. Comput."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"3049","DOI":"10.1007\/s11517-020-02251-4","article-title":"Fast Body Part Segmentation and Tracking of Neonatal Video Data using Deep Learning","volume":"58","author":"Ferreira","year":"2020","journal-title":"Med. Biol. Eng. Comput."},{"key":"ref_26","unstructured":"Bochkovskiy, A. (2020, December 02). Yolo_mark\u2014Github Repository. Available online: https:\/\/github.com\/AlexeyAB\/Yolo_mark."},{"key":"ref_27","unstructured":"Bochkovskiy, A., Wang, C.Y., and Liao, H.Y.M. (2020). YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv."},{"key":"ref_28","unstructured":"Redmon, J., and Farhadi, A. (2018). YOLOv3: An Incremental Improvement. arXiv."},{"key":"ref_29","unstructured":"Bochkovskiy, A. (2020, December 02). Darknet\u2014Github Repository. Available online: https:\/\/github.com\/AlexeyAB\/darknet."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Bigun, J., and Gustavsson, T. (2003). Two-Frame Motion Estimation Based on Polynomial Expansion. Image Analysis, Springer.","DOI":"10.1007\/3-540-45103-X"},{"key":"ref_31","unstructured":"Longo, D., Fauci, A., Kasper, D., Hauser, S., Jameson, J., and Loscalzo, J. (2011). Harrison\u2019s Principles of Internal Medicine, McGraw-Hill Education Ltd.. [18th ed.]."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"El Ahmar, W.A., Erlik Nowruzi, F., and Laganiere, R. (2020, January 14\u201319). Fast Human Head and Shoulder Detection Using Convolutional Networks and RGBD Data. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA.","DOI":"10.1109\/CVPRW50498.2020.00061"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Saqib, M., Khan, S., Sharma, N., and Blumenstein, M. (2018, January 8\u201313). Person Head Detection in Multiple Scales Using Deep Convolutional Neural Networks. Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil.","DOI":"10.1109\/IJCNN.2018.8489367"},{"key":"ref_34","first-page":"097002","article-title":"Clinical evaluation of fever-screening thermography: Impact of consensus guidelines and facial measurement location","volume":"25","author":"Zhou","year":"2020","journal-title":"J. Biomed. Opt."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/4\/1495\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:27:33Z","timestamp":1760160453000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/4\/1495"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,21]]},"references-count":34,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2021,2]]}},"alternative-id":["s21041495"],"URL":"https:\/\/doi.org\/10.3390\/s21041495","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,21]]}}}