{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T10:13:56Z","timestamp":1775211236401,"version":"3.50.1"},"reference-count":30,"publisher":"Springer Science and Business Media LLC","issue":"12","license":[{"start":{"date-parts":[[2020,10,23]],"date-time":"2020-10-23T00:00:00Z","timestamp":1603411200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,10,23]],"date-time":"2020-10-23T00:00:00Z","timestamp":1603411200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"DFG","award":["LE 817\/26-1"],"award-info":[{"award-number":["LE 817\/26-1"]}]},{"name":"DFG","award":["LE 817\/32-1"],"award-info":[{"award-number":["LE 817\/32-1"]}]},{"name":"BMBF","award":["01DQ17008"],"award-info":[{"award-number":["01DQ17008"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Med Biol Eng Comput"],"published-print":{"date-parts":[[2020,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Photoplethysmography imaging (PPGI) for non-contact monitoring of preterm infants in the neonatal intensive care unit (NICU) is a promising technology, as it could reduce medical adhesive-related skin injuries and associated complications. For practical implementations of PPGI, a region of interest has to be detected automatically in real time. As the neonates\u2019 body proportions differ significantly from adults, existing approaches may not be used in a straightforward way, and color-based skin detection requires RGB data, thus prohibiting the use of less-intrusive near-infrared (NIR) acquisition. In this paper, we present a deep learning-based method for segmentation of neonatal video data. We augmented an existing encoder-decoder semantic segmentation method with a modified version of the ResNet-50 encoder. This reduced the computational time by a factor of 7.5, so that 30 frames per second can be processed at 960 \u00d7 576 pixels. The method was developed and optimized on publicly available databases with segmentation data from adults. For evaluation, a comprehensive dataset consisting of RGB and NIR video recordings from 29 neonates with various skin tones recorded in two NICUs in Germany and India was used. From all recordings, 643 frames were manually segmented. After pre-training the model on the public adult data, parts of the neonatal data were used for additional learning and left-out neonates are used for cross-validated evaluation. On the RGB data, the head is segmented well (82% intersection over union, 88% accuracy), and performance is comparable with those achieved on large, public, non-neonatal datasets. On the other hand, performance on the NIR data was inferior. By employing data augmentation to generate additional virtual NIR data for training, results could be improved and the head could be segmented with 62% intersection over union and 65% accuracy. The method is in theory capable of performing segmentation in real time and thus it may provide a useful tool for future PPGI applications.<\/jats:p>","DOI":"10.1007\/s11517-020-02251-4","type":"journal-article","created":{"date-parts":[[2020,10,22]],"date-time":"2020-10-22T19:03:52Z","timestamp":1603393432000},"page":"3049-3061","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":24,"title":["Fast body part segmentation and tracking of neonatal video data using deep learning"],"prefix":"10.1007","volume":"58","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7948-8181","authenticated-orcid":false,"given":"Christoph Hoog","family":"Antink","sequence":"first","affiliation":[]},{"given":"Joana Carlos Mesquita","family":"Ferreira","sequence":"additional","affiliation":[]},{"given":"Michael","family":"Paul","sequence":"additional","affiliation":[]},{"given":"Simon","family":"Lyra","sequence":"additional","affiliation":[]},{"given":"Konrad","family":"Heimann","sequence":"additional","affiliation":[]},{"given":"Srinivasa","family":"Karthik","sequence":"additional","affiliation":[]},{"given":"Jayaraj","family":"Joseph","sequence":"additional","affiliation":[]},{"given":"Kumutha","family":"Jayaraman","sequence":"additional","affiliation":[]},{"given":"Thorsten","family":"Orlikowsky","sequence":"additional","affiliation":[]},{"given":"Mohanasankar","family":"Sivaprakasam","sequence":"additional","affiliation":[]},{"given":"Steffen","family":"Leonhardt","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2020,10,23]]},"reference":[{"issue":"9832","key":"2251_CR1","doi-asserted-by":"publisher","first-page":"2162","DOI":"10.1016\/S0140-6736(12)60820-4","volume":"379","author":"H Blencowe","year":"2012","unstructured":"Blencowe H, Cousens S, Oestergaard MZ, Chou D, Moller A-B, Narwal R, Adler A, Vera Garcia C, Rohde S, Say L, Lawn JE (2012) National, regional, and worldwide estimates of preterm birth rates in the year 2010 with time trends since 1990 for selected countries: a systematic analysis and implications. Lancet 379(9832):2162\u20132172. https:\/\/doi.org\/10.1016\/S0140-6736(12)60820-4","journal-title":"Lancet"},{"key":"2251_CR2","volume-title":"Preterm birth: causes, consequences and prevention","author":"AS Butler","year":"2007","unstructured":"Butler AS, Behrman RE (2007) Preterm birth: causes, consequences and prevention, 1st edn. National Academies Press (US), Washington","edition":"1st edn."},{"key":"2251_CR3","doi-asserted-by":"publisher","unstructured":"Jorge J, Villarroel M, Chaichulee S, Guazzi A, Davis S, Green G, McCormick K, Tarassenko L (2017) Non-contact monitoring of respiration in the neonatal intensive care unit. In: 2017 12Th IEEE international conference on automatic face and gesture recognition (FG 2017). https:\/\/doi.org\/10.1109\/FG.2017.44. http:\/\/ieeexplore.ieee.org\/document\/7961754\/. IEEE, pp 286\u2013293","DOI":"10.1109\/FG.2017.44"},{"issue":"4","key":"2251_CR4","doi-asserted-by":"publisher","first-page":"160","DOI":"10.1053\/j.nainr.2014.10.001","volume":"14","author":"C Lund","year":"2014","unstructured":"Lund C (2014) Medical adhesives in the NICU. Newborn Infant Nurs Rev 14(4):160\u2013165. https:\/\/doi.org\/10.1053\/j.nainr.2014.10.001","journal-title":"Newborn Infant Nurs Rev"},{"key":"2251_CR5","doi-asserted-by":"publisher","unstructured":"Wu T, Blazek V, Schmitt HJ (2000) Photoplethysmography imaging: a new noninvasive and noncontact method for mapping of the dermal perfusion changes. In: Priezzhev AV, Oberg PA (eds) Proceedings of SPIE, vol 4163, p 62. https:\/\/doi.org\/10.1117\/12.407646. http:\/\/proceedings.spiedigitallibrary.org\/proceeding.aspx?doi=10.1117\/12.407646","DOI":"10.1117\/12.407646"},{"issue":"5","key":"2251_CR6","doi-asserted-by":"publisher","first-page":"617","DOI":"10.1515\/bmt-2017-0119","volume":"63","author":"S Zaunseder","year":"2018","unstructured":"Zaunseder S, Trumpp A, Wedekind D, Malberg H (2018) Cardiovascular assessment by imaging photoplethysmography \u2013 a review. Biomed Eng \/ Biomedizinische Technik 63(5):617\u2013634. 10.1515\/bmt-2017-0119","journal-title":"Biomed Eng \/ Biomedizinische Technik"},{"issue":"01","key":"2251_CR7","doi-asserted-by":"publisher","first-page":"102","DOI":"10.1055\/s-0039-1677914","volume":"28","author":"C Hoog Antink","year":"2019","unstructured":"Hoog Antink C, Lyra S, Paul M, Yu X, Leonhardt S (2019) A broader look: camera-Based vital sign estimation across the spectrum. Yearb MedInform 28(01):102\u2013114. https:\/\/doi.org\/10.1055\/s-0039-1677914","journal-title":"Yearb MedInform"},{"key":"2251_CR8","doi-asserted-by":"publisher","unstructured":"Scalise L, Bernacchia N, Ercoli I, Marchionni P (2012) Heart rate measurement in neonatal patients using a webcamera. In: 2012 IEEE international symposium on medical measurements and applications proceedings. IEEE, pp 1\u20134. https:\/\/doi.org\/10.1109\/MeMeA.2012.6226654. http:\/\/ieeexplore.ieee.org\/document\/6226654\/","DOI":"10.1109\/MeMeA.2012.6226654"},{"key":"2251_CR9","doi-asserted-by":"publisher","unstructured":"Chaichulee S, Villarroel M, Jorge J, Arteta C, Green G, McCormick K, Zisserman A, Tarassenko L (2017) Multi-task convolutional neural network for patient detection and skin segmentation in continuous non-contact vital sign monitoring. In: 2017 12th IEEE international conference on automatic face and gesture recognition (FG 2017). IEEE, pp 266\u2013272. https:\/\/doi.org\/10.1109\/FG.2017.41. http:\/\/ieeexplore.ieee.org\/document\/7961751\/","DOI":"10.1109\/FG.2017.41"},{"issue":"6","key":"2251_CR10","doi-asserted-by":"publisher","first-page":"631","DOI":"10.1515\/bmt-2016-0025","volume":"61","author":"N Blanik","year":"2016","unstructured":"Blanik N, Heimann K, Pereira C, Paul M, Blazek V, Venema B, Orlikowsky T, Leonhardt S (2016) Remote vital parameter monitoring in neonatology - robust, unobtrusive heart rate detection in a realistic clinical scenario. Biomed Eng \/ Biomedizinische Technik 61(6):631\u2013643. https:\/\/doi.org\/10.1515\/bmt-2016-0025","journal-title":"Biomed Eng \/ Biomedizinische Technik"},{"issue":"12","key":"2251_CR11","doi-asserted-by":"publisher","first-page":"4362","DOI":"10.3390\/s18124362","volume":"18","author":"J-C Cobos-Torres","year":"2018","unstructured":"Cobos-Torres J-C, Abderrahim M, Mart\u00ednez-Orgado J (2018) Non-contact, simple neonatal monitoring by photoplethysmography. Sensors 18(12):4362. https:\/\/doi.org\/10.3390\/s18124362","journal-title":"Sensors"},{"key":"2251_CR12","doi-asserted-by":"publisher","unstructured":"Antognoli L, Marchionni P, Nobile S, Carnielli V, Scalise L (2018) Assessment of cardio-respiratory rates by non-invasive measurement methods in hospitalized preterm neonates. In: 2018 IEEE international symposium on Medical Measurements and Applications (MEMEA). https:\/\/ieeexplore.ieee.org\/document\/8438772\/. IEEE, pp 1\u20135, DOI https:\/\/doi.org\/10.1109\/MeMeA.2018.8438772, (to appear in print)","DOI":"10.1109\/MeMeA.2018.8438772"},{"issue":"12","key":"2251_CR13","doi-asserted-by":"publisher","first-page":"943","DOI":"10.1016\/j.earlhumdev.2013.09.016","volume":"89","author":"LAM Aarts","year":"2013","unstructured":"Aarts LAM, Jeanne V, Cleary JP, Lieber C, Nelson JS, Bambang Oetomo S, Verkruysse W (2013) Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit - a pilot study. Early Hum Dev 89(12):943\u2013948. https:\/\/doi.org\/10.1016\/j.earlhumdev.2013.09.016","journal-title":"Early Hum Dev"},{"issue":"3","key":"2251_CR14","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1049\/htl.2014.0077","volume":"1","author":"S Davis","year":"2014","unstructured":"Davis S, Watkinson P, Guazzi A, McCormick K, Tarassenko L, Jorge J, Villarroel M, Shenvi A, Green G (2014) Continuous non-contact vital sign monitoring in neonatal intensive care unit. Healthc Technol Lett 1(3):87\u201391. https:\/\/doi.org\/10.1049\/htl.2014.0077","journal-title":"Healthc Technol Lett"},{"key":"2251_CR15","doi-asserted-by":"publisher","unstructured":"Sikdar A, Behera SK, Dogra DP, Bhaskar H (2015) Contactless vision-based pulse rate detection of infants under neurological examinations. In: 2015 37th annual international conference of the IEEE engineering in medicine and biology society (EMBC). https:\/\/doi.org\/10.1109\/EMBC.2015.7318446. https:\/\/ieeexplore.ieee.org\/document\/7318446\/, vol 2015-novem. IEEE, pp 650\u2013653","DOI":"10.1109\/EMBC.2015.7318446"},{"key":"2251_CR16","doi-asserted-by":"publisher","unstructured":"Green G, Chaichulee S, Villarroel M, Jorge J, Arteta C, Zisserman A, Tarassenko L, McCormick K (2018) Localised photoplethysmography imaging for heart rate estimation of pre-term infants in the clinic. In: Cot\u00e9 GL (ed) Optical diagnostics and sensing XVIII: toward point-of-care diagnostics. SPIE, p 26. https:\/\/doi.org\/10.1117\/12.2289759. https:\/\/doi.org\/https:\/\/www.spiedigitallibrary.org\/conference-proceedings-of-spie\/10501\/2289759\/Localised-photoplethysmography-imaging-for-heart-rate-estimation-of-pre-term\/10.1117\/12.2289759.full","DOI":"10.1117\/12.2289759"},{"key":"2251_CR17","doi-asserted-by":"publisher","unstructured":"Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 3431\u20133440. https:\/\/doi.org\/10.1109\/CVPR.2015.7298965. https:\/\/people.eecs.berkeley.edu\/%7B~%7Djonlong\/long_shelhamer_fcn.pdfhttp:\/\/ieeexplore.ieee.org\/document\/7298965\/","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"2251_CR18","unstructured":"Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, pp 91\u201399. http:\/\/image-net.org\/challenges\/LSVRC\/2015\/results. arXiv:1506.01497"},{"key":"2251_CR19","unstructured":"Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556"},{"issue":"2","key":"2251_CR20","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1007\/s11263-009-0275-4","volume":"88","author":"M Everingham","year":"2010","unstructured":"Everingham M, Van Gool L, Williams CKI, Winn J, Zisserman A (2010) The pascal visual object classes (VOC) challenge. Int J Comput Vis 88(2):303\u2013338. https:\/\/doi.org\/10.1007\/s11263-009-0275-4","journal-title":"Int J Comput Vis"},{"key":"2251_CR21","doi-asserted-by":"publisher","unstructured":"Oliveira GL, Valada A, Bollen C, Burgard W, Brox T (2016) Deep learning for human part discovery in images. In: 2016 IEEE International conference on robotics and automation (ICRA), vol 2016-june. IEEE, pp 1634\u20131641. https:\/\/doi.org\/10.1109\/ICRA.2016.7487304. http:\/\/ieeexplore.ieee.org\/document\/7487304\/","DOI":"10.1109\/ICRA.2016.7487304"},{"issue":"4","key":"2251_CR22","first-page":"00915","volume":"1606","author":"L-C Chen","year":"2017","unstructured":"Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille A (2017) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Patterb Anal Mach Intel 1606(4):00915","journal-title":"IEEE Trans Patterb Anal Mach Intel"},{"key":"2251_CR23","doi-asserted-by":"crossref","unstructured":"Islam MA, Rochan M, Naha S, Bruce NDB, Wang Y (2018) Gated feedback refinement network for coarse-to-fine dense semantic image labeling. arXiv:1806.11266","DOI":"10.1109\/CVPR.2017.518"},{"key":"2251_CR24","doi-asserted-by":"publisher","unstructured":"Lin G, Milan A, Shen C, Reid I (2017) RefineNet: multi-path refinement networks for high-resolution semantic segmentation. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https:\/\/doi.org\/10.1109\/CVPR.2017.549. arXiv:1611.06612v3. http:\/\/ieeexplore.ieee.org\/document\/8100032\/, vol 2017-Janua. IEEE, pp 5168\u20135177","DOI":"10.1109\/CVPR.2017.549"},{"issue":"2","key":"2251_CR25","doi-asserted-by":"publisher","first-page":"024001","DOI":"10.1088\/1361-6579\/ab755c","volume":"41","author":"M Paul","year":"2020","unstructured":"Paul M, Karthik S, Joseph J, Sivaprakasam M, Kumutha J, Leonhardt S, Hoog Antink C (2020) Non-contact sensing of neonatal pulse rate using camera-based imaging: a clinical feasibility study. Physiol Meas 41(2):024001. https:\/\/doi.org\/10.1088\/1361-6579\/ab755c","journal-title":"Physiol Meas"},{"key":"2251_CR26","doi-asserted-by":"publisher","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol abs\/1512.0. IEEE, pp 770\u2013778. https:\/\/doi.org\/10.1109\/CVPR.2016.90.1512.03385. http:\/\/image-net.org\/challenges\/LSVRC\/2015\/. arXiv:1512.03385. http:\/\/ieeexplore.ieee.org\/document\/7780459\/","DOI":"10.1109\/CVPR.2016.90.1512.03385"},{"key":"2251_CR27","doi-asserted-by":"publisher","unstructured":"Deng J, Dong W, Socher R, Li L-J, Li K, Li F-F (2009) ImageNet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. IEEE, pp 248\u2013255. https:\/\/doi.org\/10.1109\/CVPR.2009.5206848. http:\/\/www.image-net.org. https:\/\/ieeexplore.ieee.org\/document\/5206848\/","DOI":"10.1109\/CVPR.2009.5206848"},{"issue":"1","key":"2251_CR28","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1186\/s40537-019-0197-0","volume":"6","author":"C Shorten","year":"2019","unstructured":"Shorten C, Khoshgoftaar TM (2019) A survey on image data augmentation for deep learning. J Big Data 6(1):60. https:\/\/doi.org\/10.1186\/s40537-019-0197-0","journal-title":"J Big Data"},{"key":"2251_CR29","doi-asserted-by":"publisher","unstructured":"Chen X, Mottaghi R, Liu X, Fidler S, Urtasun R, Yuille A (2014) Detect what you can: detecting and representing objects using holistic models and body parts. IEEE, pp 1979\u20131986. https:\/\/doi.org\/10.1109\/CVPR.2014.254.1406.2031. http:\/\/ieeexplore.ieee.org\/lpdocs\/epic03\/wrapper.htm?arnumber=6909651","DOI":"10.1109\/CVPR.2014.254.1406.2031"},{"key":"2251_CR30","unstructured":"Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. In: 3rd international conference on learning representations, ICLR 2015 - Conference track proceedings, pp 1\u201315. arXiv:1412.6980"}],"container-title":["Medical &amp; Biological Engineering &amp; Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11517-020-02251-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11517-020-02251-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11517-020-02251-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,10,22]],"date-time":"2021-10-22T20:26:47Z","timestamp":1634934407000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11517-020-02251-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,23]]},"references-count":30,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2020,12]]}},"alternative-id":["2251"],"URL":"https:\/\/doi.org\/10.1007\/s11517-020-02251-4","relation":{},"ISSN":["0140-0118","1741-0444"],"issn-type":[{"value":"0140-0118","type":"print"},{"value":"1741-0444","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,10,23]]},"assertion":[{"value":"29 March 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 August 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 October 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"A dataset recorded in two different hospital settings was used in this work. The first subset was recorded at RWTH Aachen University Hospital (UKA), Department of Neonatology, Aachen, Germany (Aachen subset), and the study was approved by the ethics committee of the UKA, Aachen, Germany (EK 327\/16). The second subset (Chennai subset) was recorded at Saveetha Medical College and Hospital, Chennai, India, and the study was approved by the institutional ethics committee of Saveetha University (SMC\/IEC\/2018\/03\/067).","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Compliance with ethical standards"}},{"value":"The authors declare that they have no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"<!--Emphasis Type='Bold' removed-->Competing interests"}}]}}