{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,7]],"date-time":"2026-04-07T16:13:45Z","timestamp":1775578425584,"version":"3.50.1"},"reference-count":55,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2016,5,18]],"date-time":"2016-05-18T00:00:00Z","timestamp":1463529600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Technicolor"},{"name":"ERC Starting Grant CapReal"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2016,6,2]]},"abstract":"<jats:p>We present a novel approach for the automatic creation of a personalized high-quality 3D face rig of an actor from just monocular video data (e.g., vintage movies). Our rig is based on three distinct layers that allow us to model the actor\u2019s facial shape as well as capture his person-specific expression characteristics at high fidelity, ranging from coarse-scale geometry to fine-scale static and transient detail on the scale of folds and wrinkles. At the heart of our approach is a parametric shape prior that encodes the plausible subspace of facial identity and expression variations. Based on this prior, a coarse-scale reconstruction is obtained by means of a novel variational fitting approach. We represent person-specific idiosyncrasies, which cannot be represented in the restricted shape and expression space, by learning a set of medium-scale corrective shapes. Fine-scale skin detail, such as wrinkles, are captured from video via shading-based refinement, and a generative detail formation model is learned. Both the medium- and fine-scale detail layers are coupled with the parametric prior by means of a novel sparse linear regression formulation. Once reconstructed, all layers of the face rig can be conveniently controlled by a low number of blendshape expression parameters, as widely used by animation artists. We show captured face rigs and their motions for several actors filmed in different monocular video formats, including legacy footage from YouTube, and demonstrate how they can be used for 3D animation and 2D video editing. Finally, we evaluate our approach qualitatively and quantitatively and compare to related state-of-the-art methods.<\/jats:p>","DOI":"10.1145\/2890493","type":"journal-article","created":{"date-parts":[[2016,5,18]],"date-time":"2016-05-18T14:28:02Z","timestamp":1463581682000},"page":"1-15","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":155,"title":["Reconstruction of Personalized 3D Face Rigs from Monocular Video"],"prefix":"10.1145","volume":"35","author":[{"given":"Pablo","family":"Garrido","sequence":"first","affiliation":[{"name":"Max-Planck-Institute for Informatics, Saarbr\u00fccken, Germany"}]},{"given":"Michael","family":"Zollh\u00f6fer","sequence":"additional","affiliation":[{"name":"Max-Planck-Institute for Informatics, Saarbr\u00fccken, Germany"}]},{"given":"Dan","family":"Casas","sequence":"additional","affiliation":[{"name":"Max-Planck-Institute for Informatics, Saarbr\u00fccken, Germany"}]},{"given":"Levi","family":"Valgaerts","sequence":"additional","affiliation":[{"name":"Max-Planck-Institute for Informatics, Saarbr\u00fccken, Germany"}]},{"given":"Kiran","family":"Varanasi","sequence":"additional","affiliation":[{"name":"Technicolor"}]},{"given":"Patrick","family":"P\u00e9rez","sequence":"additional","affiliation":[{"name":"Technicolor, Cesson S\u00e9vign\u00e9 Cedex, France"}]},{"given":"Christian","family":"Theobalt","sequence":"additional","affiliation":[{"name":"Max-Planck-Institute for Informatics, Saarbr\u00fccken, Germany"}]}],"member":"320","published-online":{"date-parts":[[2016,5,18]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/566654.566592"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1667239.1667251"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778777"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964970"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661229.2661285"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2546276"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1111\/1467-8659.t01-1-00712"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/311535.311556"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2461976"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766943"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601204"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2013.249"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1023\/B:VISI.0000025800.10423.1f"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0031-3203(02)00052-3"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2638549"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508380"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12552"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12053"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1137\/0907079"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.2307\/1271436"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298776"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964969"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2012.88"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766974"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.5555\/846276.846303"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/3DIMPVT.2012.67"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1090\/qam\/10666"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1837101.1837109"},{"key":"e_1_2_2_29_1","unstructured":"J. P. Lewis Ken Anjyo Taehyun Rhee Mengjie Zhang Fred Pighin and Zhigang Deng. 2014. Practice and theory of blendshape facial models. In Eurographics STARs. 199--218.  J. P. Lewis Ken Anjyo Taehyun Rhee Mengjie Zhang Fred Pighin and Zhigang Deng. 2014. Practice and theory of blendshape facial models. In Eurographics STARs. 199--218."},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2462019"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cad.2014.08.016"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/1409060.1409074"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1137\/0111030"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1007\/BFb0067700"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/BFb0094775"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073226"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508417"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2010.01772.x"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383271"},{"key":"e_1_2_2_40_1","volume-title":"Cohn","author":"Saragih Jason M.","year":"2011","unstructured":"Jason M. Saragih , Simon Lucey , and Jeffrey F . Cohn . 2011 . Real-time avatar animation from a single image. In Proc. FG. 213--220. Jason M. Saragih, Simon Lucey, and Jeffrey F. Cohn. 2011. Real-time avatar animation from a single image. In Proc. FG. 213--220."},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661229.2661290"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073208"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015706.1015736"},{"key":"e_1_2_2_44_1","volume-title":"Seitz","author":"Suwajanakorn Supasorn","year":"2014","unstructured":"Supasorn Suwajanakorn , Ira Kemelmacher-Shlizerman , and Steven M . Seitz . 2014 . Total moving face reconstruction. In Proc. ECCV. 796--812. Supasorn Suwajanakorn, Ira Kemelmacher-Shlizerman, and Steven M. Seitz. 2014. Total moving face reconstruction. In Proc. ECCV. 796--812."},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964971"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/2816795.2818056"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2366145.2366206"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2008.01122.x"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073209"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/1516522.1516526"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964972"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073258"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661229.2661232"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601165"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1002\/cav.1584"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2890493","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2890493","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:38:55Z","timestamp":1750221535000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2890493"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,5,18]]},"references-count":55,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2016,6,2]]}},"alternative-id":["10.1145\/2890493"],"URL":"https:\/\/doi.org\/10.1145\/2890493","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,5,18]]},"assertion":[{"value":"2015-09-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2016-01-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2016-05-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}