{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T01:09:05Z","timestamp":1777597745112,"version":"3.51.4"},"reference-count":47,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2021,12,1]],"date-time":"2021-12-01T00:00:00Z","timestamp":1638316800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:p>We propose a simple algorithm for automatic transfer of facial expressions, from videos to a 3D character, as well as between distinct 3D characters through their rendered animations. Our method begins by learning a common, semantically-consistent latent representation for the different input image domains using an unsupervised image-to-image translation model. It subsequently learns, in a supervised manner, a linear mapping from the character images' encoded representation to the animation coefficients. At inference time, given the source domain (i.e., actor footage), it regresses the corresponding animation coefficients for the target character. Expressions are automatically remapped between the source and target identities despite differences in physiognomy. We show how our technique can be used in the context of markerless motion capture with controlled lighting conditions, for one actor and for multiple actors. Additionally, we show how it can be used to automatically transfer facial animation between distinct characters without consistent mesh parameterization and without engineered geometric priors. We compare our method with standard approaches used in production and with recent state-of-the-art models on single camera face tracking.<\/jats:p>","DOI":"10.1145\/3478513.3480515","type":"journal-article","created":{"date-parts":[[2021,12,10]],"date-time":"2021-12-10T18:28:45Z","timestamp":1639160925000},"page":"1-18","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":29,"title":["Semi-supervised video-driven facial animation transfer for production"],"prefix":"10.1145","volume":"40","author":[{"given":"Lucio","family":"Moser","sequence":"first","affiliation":[{"name":"Digital Domain, Canada"}]},{"given":"Chinyu","family":"Chien","sequence":"additional","affiliation":[{"name":"Digital Domain, Taiwan"}]},{"given":"Mark","family":"Williams","sequence":"additional","affiliation":[{"name":"Digital Domain, Canada"}]},{"given":"Jose","family":"Serra","sequence":"additional","affiliation":[{"name":"Digital Domain, Canada"}]},{"given":"Darren","family":"Hendler","sequence":"additional","affiliation":[{"name":"Digital Domain"}]},{"given":"Doug","family":"Roble","sequence":"additional","affiliation":[{"name":"Digital Domain"}]}],"member":"320","published-online":{"date-parts":[[2021,12,10]]},"reference":[{"key":"e_1_2_2_1_1","unstructured":"2019. DeepFakes\/Faceswap. https:\/\/github.com\/deepfakes\/faceswap  2019. DeepFakes\/Faceswap. https:\/\/github.com\/deepfakes\/faceswap"},{"key":"e_1_2_2_2_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Abrevaya Victoria Fernandez","year":"2020","unstructured":"Victoria Fernandez Abrevaya , Adnane Boukhayma , Philip H.S. Torr , and Edmond Boyer . 2020 . Cross-Modal Deep Face Normals With Deactivable Skip Connections . In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Victoria Fernandez Abrevaya, Adnane Boukhayma, Philip H.S. Torr, and Edmond Boyer. 2020. Cross-Modal Deep Face Normals With Deactivable Skip Connections. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964970"},{"key":"e_1_2_2_4_1","unstructured":"Mario Botsch R. Sumner M. Pauly and M. Gross. 2006. Deformation Transfer for Detail-Preserving Surface Editing.  Mario Botsch R. Sumner M. Pauly and M. Gross. 2006. Deformation Transfer for Detail-Preserving Surface Editing."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.116"},{"key":"e_1_2_2_6_1","volume-title":"Neural Head Reenactment with Latent Pose Descriptors. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Burkov Egor","year":"2020","unstructured":"Egor Burkov , Igor Pasechnik , Artur Grigorev , and Victor Lempitsky . 2020 . Neural Head Reenactment with Latent Pose Descriptors. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Egor Burkov, Igor Pasechnik, Artur Grigorev, and Victor Lempitsky. 2020. Neural Head Reenactment with Latent Pose Descriptors. In IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58558-7_9"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00995"},{"key":"e_1_2_2_9_1","volume-title":"Local Geometric Indexing of High Resolution Data for Facial Reconstruction from Sparse Markers. CoRR abs\/1903.00119","author":"Cong Matthew","year":"2019","unstructured":"Matthew Cong , Lana Lan , and Ronald Fedkiw . 2019. Local Geometric Indexing of High Resolution Data for Facial Reconstruction from Sparse Markers. CoRR abs\/1903.00119 ( 2019 ). arXiv:1903.00119 http:\/\/arxiv.org\/abs\/1903.00119 Matthew Cong, Lana Lan, and Ronald Fedkiw. 2019. Local Geometric Indexing of High Resolution Data for Facial Reconstruction from Sparse Markers. CoRR abs\/1903.00119 (2019). arXiv:1903.00119 http:\/\/arxiv.org\/abs\/1903.00119"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3395208"},{"key":"e_1_2_2_11_1","doi-asserted-by":"crossref","unstructured":"P. Ekman and W. Friesen. 1978. Facial action coding system: A technique for the measurement of facial movement. Consulting Psychologists Press.  P. Ekman and W. Friesen. 1978. Facial action coding system: A technique for the measurement of facial movement. Consulting Psychologists Press.","DOI":"10.1037\/t27734-000"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459936"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459936"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2638549"},{"key":"e_1_2_2_15_1","volume-title":"Attention Mesh: High-fidelity Face Mesh Prediction in Real-time. CoRR abs\/2006.10962","author":"Grishchenko Ivan","year":"2020","unstructured":"Ivan Grishchenko , Artsiom Ablavatski , Yury Kartynnik , Karthik Raveendran , and Matthias Grundmann . 2020 . Attention Mesh: High-fidelity Face Mesh Prediction in Real-time. CoRR abs\/2006.10962 (2020). arXiv:2006.10962 https:\/\/arxiv.org\/abs\/2006.10962 Ivan Grishchenko, Artsiom Ablavatski, Yury Kartynnik, Karthik Raveendran, and Matthias Grundmann. 2020. Attention Mesh: High-fidelity Face Mesh Prediction in Real-time. CoRR abs\/2006.10962 (2020). arXiv:2006.10962 https:\/\/arxiv.org\/abs\/2006.10962"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58529-7_10"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_2_18_1","doi-asserted-by":"crossref","unstructured":"Xun Huang Ming-Yu Liu Serge Belongie and Jan Kautz. 2018. Multimodal Unsupervised Image-to-image Translation. In ECCV.  Xun Huang Ming-Yu Liu Serge Belongie and Jan Kautz. 2018. Multimodal Unsupervised Image-to-image Translation. In ECCV.","DOI":"10.1007\/978-3-030-01219-9_11"},{"key":"e_1_2_2_19_1","volume-title":"Retrieved","author":"Imaging Digital","year":"2021","unstructured":"Digital Imaging . 2021 . DI4D PRO System . Retrieved May 19, 2021 from https:\/\/di4d.com\/technology\/ Digital Imaging. 2021. DI4D PRO System. Retrieved May 19, 2021 from https:\/\/di4d.com\/technology\/"},{"key":"e_1_2_2_20_1","volume-title":"CVPR Workshop on Learning from Unlabeled Videos.","author":"Jakab Tomas","year":"2019","unstructured":"Tomas Jakab , Ankush Gupta , Hakan Bilen , and Andrea Vedaldi . 2019 . Learning Human Pose from Unaligned Data through Image Translation . In CVPR Workshop on Learning from Unlabeled Videos. Tomas Jakab, Ankush Gupta, Hakan Bilen, and Andrea Vedaldi. 2019. Learning Human Pose from Unaligned Data through Image Translation. In CVPR Workshop on Learning from Unlabeled Videos."},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00453"},{"key":"e_1_2_2_22_1","volume-title":"Kingma and Max Welling","author":"Diederik","year":"2014","unstructured":"Diederik P. Kingma and Max Welling . 2014 a. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds .). http:\/\/arxiv.org\/abs\/1312.6114 Diederik P. Kingma and Max Welling. 2014a. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http:\/\/arxiv.org\/abs\/1312.6114"},{"key":"e_1_2_2_23_1","volume-title":"Kingma and Max Welling","author":"Diederik","year":"2014","unstructured":"Diederik P. Kingma and Max Welling . 2014 b. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings . arXiv:http:\/\/arxiv.org\/abs\/1312.6114v10 [stat.ML] Diederik P. Kingma and Max Welling. 2014b. Auto-Encoding Variational Bayes. In 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings. arXiv:http:\/\/arxiv.org\/abs\/1312.6114v10 [stat.ML]"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3099564.3099581"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01317"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130800.3130813"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/3294771.3294838"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201401"},{"key":"e_1_2_2_29_1","volume-title":"Sukno","author":"Morales Araceli","year":"2021","unstructured":"Araceli Morales , Gemma Piella , and Federico M . Sukno . 2021 . Survey on 3D face reconstruction from uncalibrated images. Computer Science Review 40 (01 May 2021), 100400. https:\/\/www.sciencedirect.com\/science\/article\/pii\/S157401372100040X Araceli Morales, Gemma Piella, and Federico M. Sukno. 2021. Survey on 3D face reconstruction from uncalibrated images. Computer Science Review 40 (01 May 2021), 100400. https:\/\/www.sciencedirect.com\/science\/article\/pii\/S157401372100040X"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3084363.3085086"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14062"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00728"},{"key":"e_1_2_2_33_1","volume-title":"Luis RP, Jian Jiang, Sheng Zhang, Pingyu Wu, Bo Zhou, and Weiming Zhang.","author":"Perov Ivan","year":"2019","unstructured":"Ivan Perov , Daiheng Gao , Nikolay Chervoniy , Kunlin Liu , Sugasa Marangonda , Chris Um\u00e9 , Mr. Dpfks , Carl Shift Facenheim , Luis RP, Jian Jiang, Sheng Zhang, Pingyu Wu, Bo Zhou, and Weiming Zhang. 2019 . DeepFaceLab . https:\/\/github.com\/iperov\/DeepFaceLab Ivan Perov, Daiheng Gao, Nikolay Chervoniy, Kunlin Liu, Sugasa Marangonda, Chris Um\u00e9, Mr. Dpfks, Carl Shift Facenheim, Luis RP, Jian Jiang, Sheng Zhang, Pingyu Wu, Bo Zhou, and Weiming Zhang. 2019. DeepFaceLab. https:\/\/github.com\/iperov\/DeepFaceLab"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/1185657.1185842"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073674"},{"key":"e_1_2_2_36_1","volume-title":"Fine-Grained Head Pose Estimation Without Keypoints. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.","author":"Ruiz Nataniel","unstructured":"Nataniel Ruiz , Eunji Chong , and James M. Rehg . 2018 . Fine-Grained Head Pose Estimation Without Keypoints. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. Nataniel Ruiz, Eunji Chong, and James M. Rehg. 2018. Fine-Grained Head Pose Estimation Without Keypoints. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops."},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00795"},{"key":"e_1_2_2_38_1","volume-title":"Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. CoRR abs\/1609.05158","author":"Shi Wenzhe","year":"2016","unstructured":"Wenzhe Shi , Jose Caballero , Ferenc Husz\u00e1r , Johannes Totz , Andrew P. Aitken , Rob Bishop , Daniel Rueckert , and Zehan Wang . 2016. Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. CoRR abs\/1609.05158 ( 2016 ). arXiv:1609.05158 http:\/\/arxiv.org\/abs\/1609.05158 Wenzhe Shi, Jose Caballero, Ferenc Husz\u00e1r, Johannes Totz, Andrew P. Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. 2016. Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. CoRR abs\/1609.05158 (2016). arXiv:1609.05158 http:\/\/arxiv.org\/abs\/1609.05158"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.401"},{"key":"e_1_2_2_40_1","volume-title":"Proc. Computer Vision and Pattern Recognition (CVPR), IEEE.","author":"Thies J.","unstructured":"J. Thies , M. Zollh\u00f6fer , M. Stamminger , C. Theobalt , and M. Nie\u00dfner . 2016. Face2Face: Real-time Face Capture and Reenactment of RGB Videos . In Proc. Computer Vision and Pattern Recognition (CVPR), IEEE. J. Thies, M. Zollh\u00f6fer, M. Stamminger, C. Theobalt, and M. Nie\u00dfner. 2016. Face2Face: Real-time Face Capture and Reenactment of RGB Videos. In Proc. Computer Vision and Pattern Recognition (CVPR), IEEE."},{"key":"e_1_2_2_41_1","doi-asserted-by":"crossref","unstructured":"Anh Tuan Tran Tal Hassner Iacopo Masi and Gerard Medioni. 2017. Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Network. In Computer Vision and Pattern Recognition (CVPR).  Anh Tuan Tran Tal Hassner Iacopo Masi and Gerard Medioni. 2017. Regressing Robust and Discriminative 3D Morphable Models with a very Deep Neural Network. In Computer Vision and Pattern Recognition (CVPR).","DOI":"10.1109\/CVPR.2017.163"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.88573"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2003.819861"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925882"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICALIP.2018.8455775"},{"key":"e_1_2_2_46_1","doi-asserted-by":"crossref","unstructured":"Yuxuan Zhang Huan Ling Jun Gao Kangxue Yin Jean-Francois Lafleche Adela Barriuso Antonio Torralba and Sanja Fidler. 2021. DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort. In CVPR.  Yuxuan Zhang Huan Ling Jun Gao Kangxue Yin Jean-Francois Lafleche Adela Barriuso Antonio Torralba and Sanja Fidler. 2021. DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort. In CVPR.","DOI":"10.1109\/CVPR46437.2021.01001"},{"key":"e_1_2_2_47_1","doi-asserted-by":"crossref","unstructured":"Michael Zollh\u00f6fer Justus Thies Darek Bradley Pablo Garrido Thabo Beeler Patrick P\u00e9erez Marc Stamminger Matthias Nie\u00dfner and Christian Theobalt. 2018. State of the Art on Monocular 3D Face Reconstruction Tracking and Applications. (2018).  Michael Zollh\u00f6fer Justus Thies Darek Bradley Pablo Garrido Thabo Beeler Patrick P\u00e9erez Marc Stamminger Matthias Nie\u00dfner and Christian Theobalt. 2018. State of the Art on Monocular 3D Face Reconstruction Tracking and Applications. (2018).","DOI":"10.1111\/cgf.13382"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3478513.3480515","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3478513.3480515","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:11:49Z","timestamp":1750191109000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3478513.3480515"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12]]},"references-count":47,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["10.1145\/3478513.3480515"],"URL":"https:\/\/doi.org\/10.1145\/3478513.3480515","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12]]},"assertion":[{"value":"2021-12-10","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}