{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,22]],"date-time":"2025-12-22T18:23:37Z","timestamp":1766427817511,"version":"3.40.4"},"reference-count":45,"publisher":"Springer Science and Business Media LLC","issue":"13","license":[{"start":{"date-parts":[[2024,5,25]],"date-time":"2024-05-25T00:00:00Z","timestamp":1716595200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,5,25]],"date-time":"2024-05-25T00:00:00Z","timestamp":1716595200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Epic MegaGrants","award":["FACS DEEP LEARNING TOO"],"award-info":[{"award-number":["FACS DEEP LEARNING TOO"]}]},{"DOI":"10.13039\/501100003141","name":"Consejo Nacional de Ciencia y Tecnolog\u00eda","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100003141","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Multimed Tools Appl"],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Due to the increasing demand for virtual avatars, there has been a recent growth in the research and development of frameworks for realistic digital humans, which create a demand for realistic and adaptable facial motion capture systems. Most frameworks belong to private companies or represent high investments, which is why the creation of democratized solutions is relevant for the growth of digital human content creation. This research work proposes a facial motion capture framework for digital humans with the use of machine learning for facial codification intensity regression. The main focus is to use coded face movement intensities to generate realistic expressions on a digital human. The ablation studies performed on the regression models show that Neural Networks, using Histogram of Oriented Gradients as features, and with person-specific normalization, present overall better performance against other methods in the literature. With an RMSE of 0.052, the proposed framework offers reliable results that can be rendered in the face of a MetaHuman.<\/jats:p>","DOI":"10.1007\/s11042-024-19400-8","type":"journal-article","created":{"date-parts":[[2024,5,25]],"date-time":"2024-05-25T05:01:31Z","timestamp":1716613291000},"page":"11775-11794","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Action unit intensity regression for facial MoCap aimed towards digital humans"],"prefix":"10.1007","volume":"84","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5289-6070","authenticated-orcid":false,"given":"Carlos","family":"Vilchis","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9329-8907","authenticated-orcid":false,"given":"Mauricio","family":"Mendez-Ruiz","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7024-9376","authenticated-orcid":false,"given":"Carmina","family":"Perez-Guerrero","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6451-9109","authenticated-orcid":false,"given":"Miguel","family":"Gonzalez-Mendoza","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,5,25]]},"reference":[{"key":"19400_CR1","doi-asserted-by":"crossref","unstructured":"Krumhuber EG, Tamarit L, Roesch EB, Scherer KR (2012) Facsgen 2.0 animation software: Generating three-dimensional facs-valid facial expressions for emotion research. Emotion 12(2):351","DOI":"10.1037\/a0026632"},{"key":"19400_CR2","doi-asserted-by":"publisher","unstructured":"Parke FI (1972) Computer generated animation of faces. In: Proceedings of the ACM annual conference - vol 1. ACM \u201972, pp 451\u2013457. Association for Computing Machinery, New York, NY, USA. https:\/\/doi.org\/10.1145\/800193.569955","DOI":"10.1145\/800193.569955"},{"issue":"2","key":"19400_CR3","doi-asserted-by":"publisher","first-page":"98","DOI":"10.1109\/MRA.2012.2192811","volume":"19","author":"M Mori","year":"2012","unstructured":"Mori M, MacDorman KF, Kageki N (2012) The uncanny valley [from the field]. IEEE Robot Autom Mag 19(2):98\u2013100","journal-title":"IEEE Robot Autom Mag"},{"key":"19400_CR4","doi-asserted-by":"publisher","unstructured":"Deng Z, Noh J (2008) In: Deng Z, Neumann U (eds) Computer facial animation: a survey, pp 1\u201328. Springer, London. https:\/\/doi.org\/10.1007\/978-1-84628-907-1_1","DOI":"10.1007\/978-1-84628-907-1_1"},{"key":"19400_CR5","doi-asserted-by":"publisher","unstructured":"Ping HY, Abdullah LN, Sulaiman PS, Halin AA (2013) Computer facial animation: A review. Int J Comput Theory Eng 658\u2013662. https:\/\/doi.org\/10.7763\/ijcte.2013.v5.770","DOI":"10.7763\/ijcte.2013.v5.770"},{"key":"19400_CR6","doi-asserted-by":"publisher","unstructured":"Moser L, Hendler D, Roble D (2017) Masquerade: fine-scale details for head-mounted camera motion capture data. In: ACM SIGGRAPH 2017 Talks, pp 1\u20132. ACM, Los Angeles California. https:\/\/doi.org\/10.1145\/3084363.3085086","DOI":"10.1145\/3084363.3085086"},{"key":"19400_CR7","unstructured":"Faceware\u00a0Technologies I (2023) Faceware Website. https:\/\/facewaretech.com Accessed 22 May 2023"},{"key":"19400_CR8","unstructured":"FACEGOOD Co.,Ltd (2023) Avatary by Facegood. https:\/\/www.facegood.cc Accessed 22 May 2023"},{"key":"19400_CR9","unstructured":"Rosenberg EL, Ekman P (2020) What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS), 3rd edn. Series in Affective Science. Oxford University Press, Oxford"},{"key":"19400_CR10","unstructured":"Farnsworth B (2019) Facial Action Coding System (FACS) \u2013 A Visual Guidebook. https:\/\/imotions.com\/blog\/facial-action-coding-system\/ Accessed 22 May 2023"},{"key":"19400_CR11","unstructured":"Terzopoulos D (1995) Modeling living systems for computer vision. In: Proceedings of the 14th international joint conference on artificial intelligence - vol 1. IJCAI\u201995, pp 1003\u20131013. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA"},{"key":"19400_CR12","unstructured":"Lumsden B (2019) Digital humans: 3Lateral cracks the code for real-time facial performance. https:\/\/www.unrealengine.com\/en-US\/blog\/digital-humans-3lateral-cracks-the-code-for-real-time-facial-performance Accessed 22 May 2023"},{"key":"19400_CR13","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1016\/j.jbiomech.2019.06.012","volume":"93","author":"N Dagnes","year":"2019","unstructured":"Dagnes N, Marcolin F, Vezzetti E, Sarhan F-R, Dakp\u00e9 S, Marin F, Nonis F, Ben Mansour K (2019) Optimal marker set assessment for motion capture of 3d mimic facial movements. J Biomech 93:86\u201393. https:\/\/doi.org\/10.1016\/j.jbiomech.2019.06.012","journal-title":"J Biomech"},{"issue":"7","key":"19400_CR14","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1371\/journal.pone.0254965","volume":"16","author":"P Peng","year":"2021","unstructured":"Peng P, Portugal I, Alencar P, Cowan D (2021) A face recognition software framework based on principal component analysis. PLOS ONE 16(7):1\u201346. https:\/\/doi.org\/10.1371\/journal.pone.0254965","journal-title":"PLOS ONE"},{"key":"19400_CR15","doi-asserted-by":"publisher","unstructured":"De A, Saha A, Pal MC (2015) A human facial expression recognition model based on eigen face approach. Procedia Comput Sci 45: 282\u2013289. https:\/\/doi.org\/10.1016\/j.procs.2015.03.142. International Conference on Advanced Computing Technologies and Applications (ICACTA)","DOI":"10.1016\/j.procs.2015.03.142"},{"key":"19400_CR16","doi-asserted-by":"publisher","unstructured":"Nazir M, Jan Z, Sajjad M (2018) Facial expression recognition using histogram of oriented gradients based transformed features. Cluster Computing 21. https:\/\/doi.org\/10.1007\/s10586-017-0921-5","DOI":"10.1007\/s10586-017-0921-5"},{"key":"19400_CR17","unstructured":"Ravikumar S (2019) Lightweight markerless monocular face capture with 3d spatial priors. CoRR arXiv:1901.05355"},{"key":"19400_CR18","doi-asserted-by":"publisher","first-page":"12373","DOI":"10.1007\/s11042-019-08515-y","volume":"79","author":"J Goenetxea","year":"2020","unstructured":"Goenetxea J, Unzueta L, Dornaika F, Otaegui O (2020) Efficient deformable 3d face model tracking with limited hardware resources. Multimed Tools App 79:12373\u201312400","journal-title":"Multimed Tools App"},{"key":"19400_CR19","doi-asserted-by":"publisher","unstructured":"Somepalli MR, Charan MDS, Shruthi S, Palaniswamy S (2021) Implementation of single camera markerless facial motion capture using blendshapes. In: 2021 IEEE International conference on computation system and information technology for sustainable solutions (CSITSS), pp 1\u20136. https:\/\/doi.org\/10.1109\/CSITSS54238.2021.9683460","DOI":"10.1109\/CSITSS54238.2021.9683460"},{"key":"19400_CR20","doi-asserted-by":"publisher","unstructured":"Hasan MK, Ahsan MS, Abdullah-Al-Mamun, Newaz SHS, Lee GM (2021) Human face detection techniques: A comprehensive review and future research directions. Electronics 10(19). https:\/\/doi.org\/10.3390\/electronics10192354","DOI":"10.3390\/electronics10192354"},{"key":"19400_CR21","unstructured":"Minaee S, Luo P, Lin Z, Bowyer K (2021) Going deeper into face detection: A survey. arXiv:2103.14983 [cs.CV]"},{"key":"19400_CR22","doi-asserted-by":"crossref","unstructured":"Yang S, Luo P, Loy CC, Tang X (2015) WIDER FACE: A face detection benchmark. CoRR arXiv:1511.06523","DOI":"10.1109\/CVPR.2016.596"},{"key":"19400_CR23","unstructured":"Anand S (2023) Face Recognition with OpenCV. https:\/\/docs.opencv.org\/3.4\/da\/d60\/tutorial_face_main.html Accessed 22 May 2023"},{"key":"19400_CR24","doi-asserted-by":"crossref","unstructured":"Deng J, Guo J, Zhou Y, Yu J, Kotsia I, Zafeiriou S (2019) Retinaface: Single-stage dense face localisation in the wild. ArXiv arXiv:1905.00641","DOI":"10.1109\/CVPR42600.2020.00525"},{"key":"19400_CR25","doi-asserted-by":"crossref","unstructured":"Zhang S, Zhu X, Lei Z, Shi H, Wang X, Li SZ (2017) Faceboxes: A CPU real-time face detector with high accuracy. arXiv arXiv:1708.05234","DOI":"10.1109\/BTAS.2017.8272675"},{"key":"19400_CR26","unstructured":"NVIDIA NGC Catalog (2023) FaceDetect. https:\/\/catalog.ngc.nvidia.com\/orgs\/nvidia\/teams\/tao\/models\/facenet Accessed 22 May 2023"},{"key":"19400_CR27","doi-asserted-by":"publisher","unstructured":"B\u00e4tz A, De A, Palacio M, Mishra G, Rodr\u0131guez I, Pranti M, Satorre\u00a0Mulet P, Supervisor P, Shahriar MH, Lippert C (2021) Facial Landmarks Detection : A Brief Chronological Survey & Practical Implementation. https:\/\/doi.org\/10.13140\/RG.2.2.36199.98722","DOI":"10.13140\/RG.2.2.36199.98722"},{"key":"19400_CR28","doi-asserted-by":"publisher","unstructured":"Sagonas C, Tzimiropoulos G, Zafeiriou S, Pantic M (2013) 300 faces in-the-wild challenge: The first facial landmark localization challenge. In: 2013 IEEE International conference on computer vision workshops, pp 397\u2013403. https:\/\/doi.org\/10.1109\/ICCVW.2013.59","DOI":"10.1109\/ICCVW.2013.59"},{"key":"19400_CR29","unstructured":"Mallick S (2018) Facemark : Facial Landmark Detection using OpenCV. https:\/\/learnopencv.com\/facemark-facial-landmark-detection-using-opencv\/ Accessed 22 May 2023"},{"key":"19400_CR30","unstructured":"NVIDIA NGC Catalog (2023) Facial Landmark Estimator (FPENet) Model Card. https:\/\/catalog.ngc.nvidia.com\/orgs\/nvidia\/teams\/tao\/models\/fpenet Accessed 22 May 2023"},{"key":"19400_CR31","doi-asserted-by":"crossref","unstructured":"Guo J, Zhu X, Yang Y, Yang F, Lei Z, Li SZ (2020) Towards fast, accurate and stable 3d dense face alignment. CoRR arXiv:2009.09960","DOI":"10.1007\/978-3-030-58529-7_10"},{"key":"19400_CR32","unstructured":"OpenMMLab project (2023) MMPose. GitHub"},{"key":"19400_CR33","unstructured":"Grishchenko I, Bazarevsky V (2020) MediaPipe Holistic \u2014 Simultaneous Face, Hand and Pose Prediction, on Device. https:\/\/ai.googleblog.com\/2020\/12\/mediapipe-holistic-simultaneous-face.html Accessed 10 Dec 2020"},{"key":"19400_CR34","doi-asserted-by":"crossref","unstructured":"Jin H, Liao S, Shao L (2020) Pixel-in-pixel net: Towards efficient facial landmark detection in the wild. CoRR arXiv:2003.03771","DOI":"10.1007\/s11263-021-01521-4"},{"key":"19400_CR35","unstructured":"S\u00e1nchez-Lozano E, Tzimiropoulos G, Valstar MF (2018) Joint action unit localisation and intensity estimation through heatmap regression. CoRR arXiv:1805.03487"},{"key":"19400_CR36","doi-asserted-by":"publisher","unstructured":"Fan Y, Shen J, Cheng H, Tian F (2020) Joint facial action unit intensity prediction and region localisation. In: 2020 IEEE International conference on multimedia and expo (ICME), pp 1\u20136. https:\/\/doi.org\/10.1109\/ICME46284.2020.9102833","DOI":"10.1109\/ICME46284.2020.9102833"},{"key":"19400_CR37","doi-asserted-by":"crossref","unstructured":"Ma J, Li X, Ren Y, Yang R, Zhao Q (2021) Landmark-based facial feature construction and action unit intensity prediction. Mathematical Problems in Engineering 2021","DOI":"10.1155\/2021\/6623239"},{"key":"19400_CR38","doi-asserted-by":"crossref","unstructured":"Aneja D, McDuff D, Shah S (2019) A high-fidelity open embodied avatar with lip syncing and expression capabilities. In: 2019 International conference on multimodal interaction, pp 69\u201373","DOI":"10.1145\/3340555.3353744"},{"key":"19400_CR39","unstructured":"International Epic Games (2023) Epic Games MetaHuman Creator. https:\/\/metahuman.unrealengine.com\/"},{"key":"19400_CR40","doi-asserted-by":"crossref","unstructured":"Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: 2005 IEEE Computer society conference on computer vision and pattern recognition (CVPR\u201905), vol 1, pp 886\u2013893. Ieee","DOI":"10.1109\/CVPR.2005.177"},{"issue":"9","key":"19400_CR41","doi-asserted-by":"publisher","first-page":"1627","DOI":"10.1109\/TPAMI.2009.167","volume":"32","author":"PF Felzenszwalb","year":"2010","unstructured":"Felzenszwalb PF, Girshick RB, McAllester D, Ramanan D (2010) Object detection with discriminatively trained part-based models. IEEE Trans Pattern Anal Mach Intell 32(9):1627\u20131645","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"19400_CR42","doi-asserted-by":"crossref","unstructured":"Ma C, Chen L, Yong J (2019) Au r-cnn: Encoding expert prior knowledge into r-cnn for action unit detection. neurocomputing 355:35\u201347","DOI":"10.1016\/j.neucom.2019.03.082"},{"key":"19400_CR43","doi-asserted-by":"crossref","unstructured":"Baltru\u0161aitis T, Mahmoud M, Robinson P (2015) Cross-dataset learning and person-specific normalisation for automatic action unit detection. In: 2015 11th IEEE International conference and workshops on automatic face and gesture recognition (FG), vol 6, pp 1\u20136. IEEE","DOI":"10.1109\/FG.2015.7284869"},{"issue":"1","key":"19400_CR44","doi-asserted-by":"publisher","first-page":"98","DOI":"10.1109\/MSP.2008.930649","volume":"26","author":"Z Wang","year":"2009","unstructured":"Wang Z, Bovik AC (2009) Mean squared error: Love it or leave it? a new look at signal fidelity measures. IEEE Signal Process Mag 26(1):98\u2013117. https:\/\/doi.org\/10.1109\/MSP.2008.930649","journal-title":"IEEE Signal Process Mag"},{"issue":"2","key":"19400_CR45","doi-asserted-by":"publisher","first-page":"151","DOI":"10.1109\/T-AFFC.2013.4","volume":"4","author":"SM Mavadati","year":"2013","unstructured":"Mavadati SM, Mahoor MH, Bartlett K, Trinh P, Cohn JF (2013) Disfa: A spontaneous facial action intensity database. IEEE Trans Affect Comput 4(2):151\u2013160","journal-title":"IEEE Trans Affect Comput"}],"container-title":["Multimedia Tools and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11042-024-19400-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11042-024-19400-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11042-024-19400-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,2]],"date-time":"2025-05-02T03:26:28Z","timestamp":1746156388000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11042-024-19400-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,25]]},"references-count":45,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2025,4]]}},"alternative-id":["19400"],"URL":"https:\/\/doi.org\/10.1007\/s11042-024-19400-8","relation":{},"ISSN":["1573-7721"],"issn-type":[{"type":"electronic","value":"1573-7721"}],"subject":[],"published":{"date-parts":[[2024,5,25]]},"assertion":[{"value":"9 November 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 March 2024","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 May 2024","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 May 2024","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval"}}]}}