{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,18]],"date-time":"2025-11-18T12:14:24Z","timestamp":1763468064906,"version":"3.41.0"},"reference-count":39,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2011,12,1]],"date-time":"2011-12-01T00:00:00Z","timestamp":1322697600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000121","name":"Division of Mathematical Sciences","doi-asserted-by":"publisher","award":["PHY-0835713DMS-0739255"],"award-info":[{"award-number":["PHY-0835713DMS-0739255"]}],"id":[{"id":"10.13039\/100000121","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000166","name":"Division of Physics","doi-asserted-by":"publisher","award":["PHY-0835713DMS-0739255"],"award-info":[{"award-number":["PHY-0835713DMS-0739255"]}],"id":[{"id":"10.13039\/100000166","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2011,12]]},"abstract":"<jats:p>We present a method for replacing facial performances in video. Our approach accounts for differences in identity, visual appearance, speech, and timing between source and target videos. Unlike prior work, it does not require substantial manual operation or complex acquisition hardware, only single-camera video. We use a 3D multilinear model to track the facial performance in both videos. Using the corresponding 3D geometry, we warp the source to the target face and retime the source to match the target performance. We then compute an optimal seam through the video volume that maintains temporal consistency in the final composite. We showcase the use of our method on a variety of examples and present the result of a user study that suggests our results are difficult to distinguish from real video footage.<\/jats:p>","DOI":"10.1145\/2070781.2024164","type":"journal-article","created":{"date-parts":[[2011,11,30]],"date-time":"2011-11-30T13:58:46Z","timestamp":1322661526000},"page":"1-10","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":117,"title":["Video face replacement"],"prefix":"10.1145","volume":"30","author":[{"given":"Kevin","family":"Dale","sequence":"first","affiliation":[{"name":"Harvard University"}]},{"given":"Kalyan","family":"Sunkavalli","sequence":"additional","affiliation":[{"name":"Harvard University"}]},{"given":"Micah K.","family":"Johnson","sequence":"additional","affiliation":[{"name":"MIT CSAIL"}]},{"given":"Daniel","family":"Vlasic","sequence":"additional","affiliation":[{"name":"Lantos Technologies"}]},{"given":"Wojciech","family":"Matusik","sequence":"additional","affiliation":[{"name":"MIT CSAIL, and Disney Research Zurich"}]},{"given":"Hanspeter","family":"Pfister","sequence":"additional","affiliation":[{"name":"Harvard University"}]}],"member":"320","published-online":{"date-parts":[[2011,12,12]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015706.1015718"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1667239.1667251"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1964921.1964970"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/1275808.1276419"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1399504.1360638"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1111\/1467-8659.t01-1-00712"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2004.00799.x"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/965400.965469"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.969114"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1833349.1778778"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/258734.258880"},{"volume-title":"Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 231--238","author":"DeCarlo D.","key":"e_1_2_2_12_1","unstructured":"DeCarlo , D. , and Metaxas , D . 1996. The integration of optical flow and deformable models with applications to human face shape and motion estimation . In Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 231--238 . DeCarlo, D., and Metaxas, D. 1996. The integration of optical flow and deformable models with applications to human face shape and motion estimation. In Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 231--238."},{"volume-title":"Proc. Computer Animation, 68--79","author":"Essa I.","key":"e_1_2_2_13_1","unstructured":"Essa , I. , Basu , S. , Darrell , T. , and Pentland , A . 1996. Modeling, tracking and interactive animation of faces and heads: Using input from video . In Proc. Computer Animation, 68--79 . Essa, I., Basu, S., Darrell, T., and Pentland, A. 1996. Modeling, tracking and interactive animation of faces and heads: Using input from video. In Proc. Computer Animation, 68--79."},{"volume-title":"Proc. British Machine Vision Conference (BMVC), 899--908","author":"Everingham M.","key":"e_1_2_2_14_1","unstructured":"Everingham , M. , Sivic , J. , and Zisserman , A . 2006. \"Hello! My name is... Buffy\" -- automatic naming of characters in TV video . In Proc. British Machine Vision Conference (BMVC), 899--908 . Everingham, M., Sivic, J., and Zisserman, A. 2006. \"Hello! My name is... Buffy\" -- automatic naming of characters in TV video. In Proc. British Machine Vision Conference (BMVC), 899--908."},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/566654.566594"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1576246.1531373"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1507149.1507182"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/280814.280822"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1866158.1866174"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1141911.1141934"},{"volume-title":"Proc. European Conf. Visual Media Production (CVMP), 127--133","author":"Jones A.","key":"e_1_2_2_21_1","unstructured":"Jones , A. , Gardner , A. , Bolas , M. , McDowall , I. , and Debevec , P . 2006. Simulating spatially varying lighting on a live performance . In Proc. European Conf. Visual Media Production (CVMP), 127--133 . Jones, A., Gardner, A., Bolas, M., McDowall, I., and Debevec, P. 2006. Simulating spatially varying lighting on a live performance. In Proc. European Conf. Visual Media Production (CVMP), 127--133."},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1731047.1731050"},{"volume-title":"Proc. European Conf. Computer Vision (ECCV), 341--353","author":"Kemelmacher-Shlizerman I.","key":"e_1_2_2_23_1","unstructured":"Kemelmacher-Shlizerman , I. , Sankar , A. , Shechtman , E. , and Seitz , S. M . 2010. Being John Malkovich . In Proc. European Conf. Computer Vision (ECCV), 341--353 . Kemelmacher-Shlizerman, I., Sankar, A., Shechtman, E., and Seitz, S. M. 2010. Being John Malkovich. In Proc. European Conf. Computer Vision (ECCV), 341--353."},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/882262.882264"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1399504.1360637"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1661412.1618521"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/1457515.1409074"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/882262.882269"},{"volume-title":"Proc. IEEE Int. Conf. Computer Vision (ICCV), 143--150","author":"Pighin F. H.","key":"e_1_2_2_29_1","unstructured":"Pighin , F. H. , Szeliski , R. , and Salesin , D . 1999. Resynthesizing facial animation through 3d model-based tracking . In Proc. IEEE Int. Conf. Computer Vision (ICCV), 143--150 . Pighin, F. H., Szeliski, R., and Salesin, D. 1999. Resynthesizing facial animation through 3d model-based tracking. In Proc. IEEE Int. Conf. Computer Vision (ICCV), 143--150."},{"key":"e_1_2_2_30_1","volume-title":"-H","author":"Rabiner L.","year":"1993","unstructured":"Rabiner , L. , and Juang , B . -H . 1993 . Fundamentals of speech recognition. Prentice-Hall , Inc., Upper Saddle River, NJ, USA. Rabiner, L., and Juang, B.-H. 1993. Fundamentals of speech recognition. Prentice-Hall, Inc., Upper Saddle River, NJ, USA."},{"key":"e_1_2_2_31_1","first-page":"1","article-title":"What's old is new again","volume":"32","author":"Robertson B.","year":"2009","unstructured":"Robertson , B. 2009 . What's old is new again . Computer Graphics World 32 , 1 . Robertson, B. 2009. What's old is new again. Computer Graphics World 32, 1.","journal-title":"Computer Graphics World"},{"key":"e_1_2_2_32_1","unstructured":"Singular Inversions Inc. 2011. FaceGen Modeller manual. www.facegen.com.  Singular Inversions Inc. 2011. FaceGen Modeller manual. www.facegen.com."},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1833349.1778862"},{"volume-title":"Proc. IEEE Int. Conf. Computer Vision (ICCV), 747--755","author":"Viola P. A.","key":"e_1_2_2_34_1","unstructured":"Viola , P. A. , and Jones , M. J . 2001. Robust real-time face detection . In Proc. IEEE Int. Conf. Computer Vision (ICCV), 747--755 . Viola, P. A., and Jones, M. J. 2001. Robust real-time face detection. In Proc. IEEE Int. Conf. Computer Vision (ICCV), 747--755."},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073209"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/1599470.1599472"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/97879.97906"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/1964921.1964955"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015706.1015759"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2070781.2024164","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2070781.2024164","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2070781.2024164","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T10:06:02Z","timestamp":1750241162000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2070781.2024164"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2011,12]]},"references-count":39,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2011,12]]}},"alternative-id":["10.1145\/2070781.2024164"],"URL":"https:\/\/doi.org\/10.1145\/2070781.2024164","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"type":"print","value":"0730-0301"},{"type":"electronic","value":"1557-7368"}],"subject":[],"published":{"date-parts":[[2011,12]]},"assertion":[{"value":"2011-12-12","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}