{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T06:11:19Z","timestamp":1778825479837,"version":"3.51.4"},"reference-count":42,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2018,7,26]],"date-time":"2018-07-26T00:00:00Z","timestamp":1532563200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004329","name":"Javna Agencija za Raziskovalno Dejavnost RS","doi-asserted-by":"publisher","award":["P2-0095 and P5-0147"],"award-info":[{"award-number":["P2-0095 and P5-0147"]}],"id":[{"id":"10.13039\/501100004329","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100005989","name":"Ministrstvo za Izobra\u017eevanje, Znanost in \u0160port","doi-asserted-by":"publisher","award":["C3330-16-529000"],"award-info":[{"award-number":["C3330-16-529000"]}],"id":[{"id":"10.13039\/501100005989","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Measurement of energy expenditure is an important tool in sport science and medicine, especially when trying to estimate the extent and intensity of physical activity. However, most approaches still rely on sensors or markers, placed directly on the body. In this paper, we present a novel approach using a fully contact-less, fully automatic method, that relies on computer vision algorithms and widely available and inexpensive imaging sensors. We rely on the estimation of the optical and scene flow to calculate Histograms of Oriented Optical Flow (HOOF) descriptors, which we subsequently augment with the Histograms of Absolute Flow Amplitude (HAFA). Descriptors are fed into regression model, which allows us to estimate energy consumption, and to a lesser extent, the heart rate. Our method has been tested both in lab environment and in realistic conditions of a sport match. Results confirm that these energy expenditures could be derived from purely contact-less observations. The proposed method can be used with different modalities, including near infrared imagery, which extends its future potential.<\/jats:p>","DOI":"10.3390\/s18082435","type":"journal-article","created":{"date-parts":[[2018,7,26]],"date-time":"2018-07-26T11:32:07Z","timestamp":1532604727000},"page":"2435","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Quantitative Contact-Less Estimation of Energy Expenditure from Video and 3D Imagery"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5426-5686","authenticated-orcid":false,"given":"Gregor","family":"Koporec","sequence":"first","affiliation":[{"name":"Faculty of Electrical Engineering, University of Ljubljana, Tr\u017ea\u0161ka Cesta 25, SI-1000 Ljubljana, Slovenia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Goran","family":"Vu\u010dkovi\u0107","sequence":"additional","affiliation":[{"name":"Faculty of Sport, University of Ljubljana, Gortanova 22, SI-1000 Ljubljana, Slovenia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4317-0494","authenticated-orcid":false,"given":"Radoje","family":"Mili\u0107","sequence":"additional","affiliation":[{"name":"Faculty of Sport, University of Ljubljana, Gortanova 22, SI-1000 Ljubljana, Slovenia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6039-6110","authenticated-orcid":false,"given":"Janez","family":"Per\u0161","sequence":"additional","affiliation":[{"name":"Faculty of Electrical Engineering, University of Ljubljana, Tr\u017ea\u0161ka Cesta 25, SI-1000 Ljubljana, Slovenia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,7,26]]},"reference":[{"key":"ref_1","first-page":"126","article-title":"Physical activity, exercise, and physical fitness: Definitions and distinctions for health-related research","volume":"100","author":"Caspersen","year":"1985","journal-title":"Public Health Rep."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"801","DOI":"10.1503\/cmaj.051351","article-title":"Health benefits of physical activity: The evidence","volume":"174","author":"Warburton","year":"2006","journal-title":"Can. Med. Assoc. J."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1193","DOI":"10.1152\/japplphysiol.00160.2005","article-title":"Epidemiological evidence for the role of physical activity in reducing risk of type 2 diabetes and cardiovascular disease","volume":"99","author":"Bassuk","year":"2005","journal-title":"J. Appl. Physiol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"3022","DOI":"10.1519\/JSC.0b013e318234e613","article-title":"Energy expenditure during tennis play: A preliminary video analysis and metabolic model approach","volume":"25","author":"Botton","year":"2011","journal-title":"J. Strength Cond. Res."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1249\/MSS.0b013e3181ae5cfd","article-title":"Energy cost and metabolic power in elite soccer: A new match analysis approach","volume":"42","author":"Osgnach","year":"2010","journal-title":"Med. Sci. Sports Exerc."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1046\/j.1365-201X.1998.0298f.x","article-title":"Energy supply and muscle fatigue in humans","volume":"162","author":"Sahlin","year":"1998","journal-title":"Acta Physiol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1080\/026404197367263","article-title":"Energetics of high-intensity exercise (soccer) with particular reference to fatigue","volume":"15","author":"Reilly","year":"1997","journal-title":"J. Sports Sci."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1186\/1550-2783-2-2-32","article-title":"Misconceptions about aerobic and anaerobic energy expenditure","volume":"2","author":"Scott","year":"2005","journal-title":"J. Int. Soc. Sports Nutr."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"134","DOI":"10.4330\/wjc.v9.i2.134","article-title":"Aerobic vs anaerobic exercise training effects on the cardiovascular system","volume":"9","author":"Patel","year":"2017","journal-title":"World J. Cardiol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1123","DOI":"10.1079\/PHN2005800","article-title":"Measurement of energy expenditure","volume":"8","author":"Levine","year":"2005","journal-title":"Public Health Nutr."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1080\/02640410470001730089","article-title":"Prediction of energy expenditure from heart rate monitoring during submaximal exercise","volume":"23","author":"Keytel","year":"2005","journal-title":"J. Sports Sci."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"314","DOI":"10.3389\/fphys.2013.00314","article-title":"Cardiovascular reactivity, stress, and physical activity","volume":"4","author":"Huang","year":"2013","journal-title":"Front. Physiol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"7772","DOI":"10.3390\/s100807772","article-title":"A review of accelerometry-based wearable motion detectors for physical activity monitoring","volume":"10","author":"Yang","year":"2010","journal-title":"Sensors"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Yang, C., Cheung, G., Chan, K., and Stankovic, V. (2014, January 14\u201318). Sleep monitoring via depth video compression & analysis. Proceedings of the 2014 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), Chengdu, China.","DOI":"10.1109\/ICMEW.2014.6890645"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"883","DOI":"10.1249\/01.MSS.0000126585.40962.22","article-title":"Improving energy expenditure estimation for physical activity","volume":"36","author":"Zhang","year":"2004","journal-title":"Med. Sci. Sports Exerc."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Thiebaud, R.S., Funk, M.D., Patton, J.C., Massey, B.L., Shay, T.E., Schmidt, M.G., and Giovannitti, N. (2018). Validity of wrist-worn consumer products to measure heart rate and energy expenditure. Digit. Health, 4.","DOI":"10.1177\/2055207618770322"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"960","DOI":"10.1016\/j.asoc.2015.05.001","article-title":"Context-based ensemble method for human energy expenditure estimation","volume":"37","author":"Gjoreski","year":"2015","journal-title":"Appl. Soft Comput."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1037","DOI":"10.1088\/0967-3334\/36\/5\/1037","article-title":"Assessing physical activity intensity by video analysis","volume":"36","author":"Silva","year":"2015","journal-title":"Physiol. Meas."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1016\/j.jvcir.2004.04.007","article-title":"Framework for measurement of the intensity of motion activity of video segments","volume":"15","author":"Peker","year":"2004","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Nathan, D., Huynh, D.Q., Rubenson, J., and Rosenberg, M. (2015). Estimating physical activity energy expenditure with the kinect sensor in an exergaming environment. PLoS ONE, 10.","DOI":"10.1371\/journal.pone.0127113"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1016\/S0167-9457(02)00096-9","article-title":"Observation and analysis of large-scale human motion","volume":"21","author":"Bon","year":"2002","journal-title":"Hum. Mov. Sci."},{"key":"ref_22","first-page":"1","article-title":"Scene flow from depth and color images","volume":"Volume 46","author":"Letouzey","year":"2011","journal-title":"Proceedings of the BMVC 2011\u2014British Machine Vision Conference"},{"key":"ref_23","unstructured":"Trucco, E., and Verri, A. (1998). Introductory Techniques for 3-D Computer Vision, Prentice Hall."},{"key":"ref_24","unstructured":"Farneb\u00e4ack, G. (July, January 29). Two-frame motion estimation based on polynomial expansion. Proceedings of the Scandinavian Conference on Image Analysis, Halmstad, Sweden."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Vedula, S., Baker, S., Rander, P., Collins, R., and Kanade, T. (1999, January 20\u201327). Three-dimensional scene flow. Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, Greece.","DOI":"10.1109\/ICCV.1999.790293"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"4275","DOI":"10.1109\/JSEN.2015.2416651","article-title":"Evaluating and Improving the Depth Accuracy of Kinect for Windows v2","volume":"15","author":"Yang","year":"2015","journal-title":"IEEE Sens. J."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Jaimez, M., Souiai, M., Gonzalez-Jimenez, J., and Cremers, D. (2015, January 26\u201330). A primal-dual framework for real-time dense RGB-D scene flow. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7138986"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Wedel, A., and Cremers, D. (2011). Stereo Scene Flow for 3D motion analysis, Springer Science & Business Media.","DOI":"10.1007\/978-0-85729-965-9"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Chaudhry, R., Ravichandran, A., Hager, G., and Vidal, R. (2009, January 20\u201325). Histograms of oriented optical flow and binet-cauchy kernels on nonlinear dynamical systems for the recognition of human actions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, Miami, FL, USA.","DOI":"10.1109\/CVPRW.2009.5206821"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1369","DOI":"10.1016\/j.patrec.2010.03.024","article-title":"Histograms of optical flow for efficient representation of body motion","volume":"31","author":"Kristan","year":"2010","journal-title":"Pattern Recognit. Lett."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"27:1","DOI":"10.1145\/1961189.1961199","article-title":"LIBSVM: A Library for Support Vector Machines","volume":"2","author":"Chang","year":"2011","journal-title":"ACM Trans. Intell. Syst. Technol."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1055","DOI":"10.1109\/72.788646","article-title":"Support vector machines for histogram-based image classification","volume":"10","author":"Chapelle","year":"1999","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_33","unstructured":"Hsu, C.-W., Chang, C.-C., and Lin, C.-J. (2018, July 26). A practical guide to support vector classification. Available online: https:\/\/www.csie.ntu.edu.tw\/~cjlin\/papers\/guide\/guide.pdf."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1959","DOI":"10.1162\/089976602760128081","article-title":"Training v-support vector regression: Theory and algorithms","volume":"14","author":"Chang","year":"2002","journal-title":"Neural Comput."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Danelljan, M., Shahbaz Khan, F., Felsberg, M., and Van de Weijer, J. (2014, January 23\u201328). Adaptive color attributes for real-time visual tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.143"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Hannuna, S., Camplani, M., Hall, J., Mirmehdi, M., Damen, D., Burghardt, T., Paiement, A., and Tao, L. (2016). DS-KCF: A real-time tracker for RGB-D data. J. Real-Time Image Process., 1\u201320.","DOI":"10.1007\/s11554-016-0654-3"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"583","DOI":"10.1109\/TPAMI.2014.2345390","article-title":"High-speed tracking with kernelized correlation filters","volume":"37","author":"Henriques","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_38","unstructured":"Xiang, L., Echtler, F., and Kerl, C. (2018, July 26). libfreenect2: Release 0.2. Available online: https:\/\/zenodo.org\/record\/50641#.W1Wpi6uYOUk."},{"key":"ref_39","unstructured":"Cerra, D. (2005). Data Mining: Practical Machine Learning Tools and Techniques, Morgan Kaufmann Publishers."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1152\/jappl.1973.34.1.128","article-title":"On-line computer analysis and breath-by-breath graphical display of exercise function tests","volume":"34","author":"Beaver","year":"1973","journal-title":"J. Appl. Physiol."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s11263-010-0390-2","article-title":"A database and evaluation methodology for optical flow","volume":"92","author":"Baker","year":"2011","journal-title":"Int. J. Comput. Vis."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1088\/0967-3334\/35\/2\/253","article-title":"Improvement of energy expenditure prediction from heart rate during running","volume":"35","author":"Charlot","year":"2014","journal-title":"Physiol. Meas."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/8\/2435\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:14:37Z","timestamp":1760195677000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/8\/2435"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,7,26]]},"references-count":42,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2018,8]]}},"alternative-id":["s18082435"],"URL":"https:\/\/doi.org\/10.3390\/s18082435","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,7,26]]}}}