{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T05:28:36Z","timestamp":1791178116116,"version":"4.1.0"},"reference-count":29,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2014,12,29]],"date-time":"2014-12-29T00:00:00Z","timestamp":1419811200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100006034","name":"University of Southern California","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100006034","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100006753","name":"U.S. Army Materiel Command","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100006753","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2014,12,29]]},"abstract":"<jats:p>We present a process for rendering a realistic facial performance with control of viewpoint and illumination. The performance is based on one or more high-quality geometry and reflectance scans of an actor in static poses, driven by one or more video streams of a performance. We compute optical flow correspondences between neighboring video frames, and a sparse set of correspondences between static scans and video frames. The latter are made possible by leveraging the relightability of the static 3D scans to match the viewpoint(s) and appearance of the actor in videos taken in arbitrary environments. As optical flow tends to compute proper correspondence for some areas but not others, we also compute a smoothed, per-pixel confidence map for every computed flow, based on normalized cross-correlation. These flows and their confidences yield a set of weighted triangulation constraints among the static poses and the frames of a performance. Given a single artist-prepared face mesh for one static pose, we optimally combine the weighted triangulation constraints, along with a shape regularization term, into a consistent 3D geometry solution over the entire performance that is drift free by construction. In contrast to previous work, even partial correspondences contribute to drift minimization, for example, where a successful match is found in the eye region but not the mouth. Our shape regularization employs a differential shape term based on a spatially varying blend of the differential shapes of the static poses and neighboring dynamic poses, weighted by the associated flow confidences. These weights also permit dynamic reflectance maps to be produced for the performance by blending the static scan maps. Finally, as the geometry and maps are represented on a consistent artist-friendly mesh, we render the resulting high-quality animated face geometry and animated reflectance maps using standard rendering tools.<\/jats:p>","DOI":"10.1145\/2638549","type":"journal-article","created":{"date-parts":[[2015,1,5]],"date-time":"2015-01-05T13:27:09Z","timestamp":1420464429000},"page":"1-14","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":72,"title":["Driving High-Resolution Facial Scans with Video Performance Capture"],"prefix":"10.1145","volume":"34","author":[{"given":"Graham","family":"Fyffe","sequence":"first","affiliation":[{"name":"University of Southern California Institute for Creative Technologies, Los Angeles, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrew","family":"Jones","sequence":"additional","affiliation":[{"name":"University of Southern California Institute for Creative Technologies, Los Angeles, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Oleg","family":"Alexander","sequence":"additional","affiliation":[{"name":"University of Southern California Institute for Creative Technologies, Los Angeles, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ryosuke","family":"Ichikari","sequence":"additional","affiliation":[{"name":"University of Southern California Institute for Creative Technologies, Los Angeles, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paul","family":"Debevec","sequence":"additional","affiliation":[{"name":"University of Southern California Institute for Creative Technologies, Los Angeles, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2014,12,29]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.5555\/1747592.1747825"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-010-0390-2"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778777"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/1964921.1964970"},{"key":"e_1_2_2_5_1","volume-title":"Proceedings of the ACM SIGGRAPH\/Eurographics Symposium on Computer Graphics and Interactive Techniques. Eurographics Association, Aire-la-Ville, Switzerland. 57--66","author":"Bickel B.","unstructured":"B. Bickel , M. Lang , M. Botsch , M. A. Otaduy , and M. Gross . 2008. Pose-space animation and transfer of facial details . In Proceedings of the ACM SIGGRAPH\/Eurographics Symposium on Computer Graphics and Interactive Techniques. Eurographics Association, Aire-la-Ville, Switzerland. 57--66 . B. Bickel, M. Lang, M. Botsch, M. A. Otaduy, and M. Gross. 2008. Pose-space animation and transfer of facial details. In Proceedings of the ACM SIGGRAPH\/Eurographics Symposium on Computer Graphics and Interactive Techniques. Eurographics Association, Aire-la-Ville, Switzerland. 57--66."},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/965400.965469"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1833349.1778778"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.927467"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/280814.280864"},{"key":"e_1_2_2_10_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'96)","author":"DeCarlo D.","unstructured":"D. DeCarlo and D. Metaxas . 1996. The integration of optical flow and deformable models with applications to human face shape and motion estimation . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'96) . 231--238. D. DeCarlo and D. Metaxas. 1996. The integration of optical flow and deformable models with applications to human face shape and motion estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'96). 231--238."},{"key":"e_1_2_2_11_1","doi-asserted-by":"crossref","unstructured":"P. Ekman and W. Friesen. 1978. Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press Palo Alto CA.  P. Ekman and W. Friesen. 1978. Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press Palo Alto CA.","DOI":"10.1037\/t27734-000"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2024156.2024163"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/280814.280822"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.5555\/2383533.2383575"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964969"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2341836.2341906"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/3DIMPVT.2012.67"},{"key":"e_1_2_2_18_1","volume-title":"Proceedings of the International Conference on 3D Imaging, Modeling, Processing, Visualization and Trasmission (3DPVT'10)","author":"Klaudiny M.","unstructured":"M. Klaudiny , A. Hilton , and J. Edge . 2010. High-detail 3d capture of facial performance . In Proceedings of the International Conference on 3D Imaging, Modeling, Processing, Visualization and Trasmission (3DPVT'10) . M. Klaudiny, A. Hilton, and J. Edge. 2010. High-detail 3d capture of facial performance. In Proceedings of the International Conference on 3D Imaging, Modeling, Processing, Visualization and Trasmission (3DPVT'10)."},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2006.200"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.216724"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1409060.1409074"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015706.1015783"},{"key":"e_1_2_2_23_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'10)","author":"Park M.","unstructured":"M. Park , S. Kashyap , R. Collins , and Y. Liu . 2010. Data driven mean-shift belief propagation for non-gaussian mrfs . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'10) . 3547--3554. M. Park, S. Kashyap, R. Collins, and Y. Liu. 2010. Data driven mean-shift belief propagation for non-gaussian mrfs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'10). 3547--3554."},{"key":"e_1_2_2_24_1","volume-title":"Proceedings of the Eurographics Symposium on Geometry Processing (SGP'10)","author":"Popa T.","unstructured":"T. Popa , I. South Ickinson , D. Bradley , A. Sheffer , and W. Heidrich . 2010. Globally consistent space-time reconstruction . In Proceedings of the Eurographics Symposium on Geometry Processing (SGP'10) . T. Popa, I. South Ickinson, D. Bradley, A. Sheffer, and W. Heidrich. 2010. Globally consistent space-time reconstruction. In Proceedings of the Eurographics Symposium on Geometry Processing (SGP'10)."},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2159516.2159519"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2366145.2366206"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/1964921.1964972"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/1186562.1015759"},{"key":"e_1_2_2_30_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'12)","author":"Zhu X.","unstructured":"X. Zhu and D. Ramanan . 2012. Face detection, pose estimation, and landmark localization in the wild . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'12) . 2879--2886. X. Zhu and D. Ramanan. 2012. Face detection, pose estimation, and landmark localization in the wild. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'12). 2879--2886."}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2638549","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2638549","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T08:10:33Z","timestamp":1750234233000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2638549"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,12,29]]},"references-count":29,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2014,12,29]]}},"alternative-id":["10.1145\/2638549"],"URL":"https:\/\/doi.org\/10.1145\/2638549","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2014,12,29]]},"assertion":[{"value":"2013-12-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2014-06-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2014-12-29","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}