{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,6]],"date-time":"2026-05-06T15:48:16Z","timestamp":1778082496705,"version":"3.51.4"},"reference-count":54,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2015,11,2]],"date-time":"2015-11-02T00:00:00Z","timestamp":1446422400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Heterogeneous Image Systems"},{"DOI":"10.13039\/501100001659","name":"German Research Foundation","doi-asserted-by":"crossref","award":["GRK-1773"],"award-info":[{"award-number":["GRK-1773"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"crossref"}]},{"name":"ERC","award":["335545"],"award-info":[{"award-number":["335545"]}]},{"name":"Max Planck Center for Visual Computing and Communications"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2015,11,4]]},"abstract":"<jats:p>We present a method for the real-time transfer of facial expressions from an actor in a source video to an actor in a target video, thus enabling the ad-hoc control of the facial expressions of the target actor. The novelty of our approach lies in the transfer and photorealistic re-rendering of facial deformations and detail into the target video in a way that the newly-synthesized expressions are virtually indistinguishable from a real video. To achieve this, we accurately capture the facial performances of the source and target subjects in real-time using a commodity RGB-D sensor. For each frame, we jointly fit a parametric model for identity, expression, and skin reflectance to the input color and depth data, and also reconstruct the scene lighting. For expression transfer, we compute the difference between the source and target expressions in parameter space, and modify the target parameters to match the source expressions. A major challenge is the convincing re-rendering of the synthesized target face into the corresponding video stream. This requires a careful consideration of the lighting and shading design, which both must correspond to the real-world environment. We demonstrate our method in a live setup, where we modify a video conference feed such that the facial expressions of a different person (e.g., translator) are matched in real-time.<\/jats:p>","DOI":"10.1145\/2816795.2818056","type":"journal-article","created":{"date-parts":[[2015,10,27]],"date-time":"2015-10-27T12:36:39Z","timestamp":1445949399000},"page":"1-14","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":303,"title":["Real-time expression transfer for facial reenactment"],"prefix":"10.1145","volume":"34","author":[{"given":"Justus","family":"Thies","sequence":"first","affiliation":[{"name":"University of Erlangen-Nuremberg"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael","family":"Zollh\u00f6fer","sequence":"additional","affiliation":[{"name":"Max-Planck-Institute for Informatics"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Matthias","family":"Nie\u00dfner","sequence":"additional","affiliation":[{"name":"Stanford University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Levi","family":"Valgaerts","sequence":"additional","affiliation":[{"name":"Max-Planck-Institute for Informatics"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marc","family":"Stamminger","sequence":"additional","affiliation":[{"name":"University of Erlangen-Nuremberg"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Christian","family":"Theobalt","sequence":"additional","affiliation":[{"name":"Max-Planck-Institute for Informatics"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2015,11,2]]},"reference":[{"key":"e_1_2_2_1_1","unstructured":"Adelson E. H. Anderson C. H. Bergen J. R. Burt P. J. and Ogden J. M. 1984. Pyramid methods in image processing. RCA engineer 29 6 33--41.  Adelson E. H. Anderson C. H. Bergen J. R. Burt P. J. and Ogden J. M. 1984. Pyramid methods in image processing. RCA engineer 29 6 33--41."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1667239.1667251"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964970"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276419"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/311535.311556"},{"key":"e_1_2_2_6_1","doi-asserted-by":"crossref","unstructured":"Blanz V. Basso C. Poggio T. and Vetter T. 2003. Reanimating faces in images and video. In Computer graphics forum Wiley Online Library 641--650.  Blanz V. Basso C. Poggio T. and Vetter T. 2003. Reanimating faces in images and video. In Computer graphics forum Wiley Online Library 641--650.","DOI":"10.1111\/1467-8659.t01-1-00712"},{"key":"e_1_2_2_7_1","volume-title":"-P","author":"Blanz V.","year":"2004","unstructured":"Blanz , V. , Scherbaum , K. , Vetter , T. , and Seidel , H . -P . 2004 . Exchanging faces in images. In Computer Graphics Forum, Wiley Online Library , 669--676. Blanz, V., Scherbaum, K., Vetter, T., and Seidel, H.-P. 2004. Exchanging faces in images. In Computer Graphics Forum, Wiley Online Library, 669--676."},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/965400.965469"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2461976"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778778"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCOM.1983.1095851"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2462012"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601204"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2013.249"},{"key":"e_1_2_2_15_1","volume-title":"Proc. SCA, Eurographics Association, 193--206","author":"Chai J.-X.","unstructured":"Chai , J.-X. , Xiao , J. , and Hodgins , J . 2003. Vision-based control of 3D facial animation . In Proc. SCA, Eurographics Association, 193--206 . Chai, J.-X., Xiao, J., and Hodgins, J. 2003. Vision-based control of 3D facial animation. In Proc. SCA, Eurographics Association, 193--206."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/0262-8856(92)90066-C"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2013.449"},{"key":"e_1_2_2_18_1","volume-title":"Tech. Rep. CS-TR-2002-02","author":"Chuang E.","year":"2002","unstructured":"Chuang , E. , and Bregler , C . 2002 . Performance-driven facial animation using blend shape interpolation. Tech. Rep. CS-TR-2002-02 , Stanford University . Chuang, E., and Bregler, C. 2002. Performance-driven facial animation using blend shape interpolation. Tech. Rep. CS-TR-2002-02, Stanford University."},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.927467"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2070781.2024164"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/38.708562"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2638549"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508380"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.537"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12552"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/280814.280822"},{"key":"e_1_2_2_27_1","doi-asserted-by":"crossref","unstructured":"Hsieh P.-L. Ma C. Yu J. and Li H. 2015. Unconstrained realtime facial performance capture. In Computer Vision and Pattern Recognition (CVPR).  Hsieh P.-L. Ma C. Yu J. and Li H. 2015. Unconstrained realtime facial performance capture. In Computer Vision and Pattern Recognition (CVPR).","DOI":"10.1109\/CVPR.2015.7298776"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964969"},{"key":"e_1_2_2_29_1","volume-title":"Proc. ECCV, 341--353","author":"Kemelmacher-Shlizerman I.","unstructured":"Kemelmacher-Shlizerman , I. , Sankar , A. , Shechtman , E. , and Seitz , S. M . 2010. Being John Malkovich . In Proc. ECCV, 341--353 . Kemelmacher-Shlizerman, I., Sankar, A., Shechtman, E., and Seitz, S. M. 2010. Being John Malkovich. In Proc. ECCV, 341--353."},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964956"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2010.41"},{"key":"e_1_2_2_32_1","volume-title":"Proc. CVPR, 57--64","author":"Li K.","unstructured":"Li , K. , Xu , F. , Wang , J. , Dai , Q. , and Liu , Y . 2012. A data-driven approach for facial expression synthesis in video . In Proc. CVPR, 57--64 . Li, K., Xu, F., Wang, J., Dai, Q., and Liu, Y. 2012. A data-driven approach for facial expression synthesis in video. In Proc. CVPR, 57--64."},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2462019"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383289"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1080\/10867651.2002.10487551"},{"key":"e_1_2_2_36_1","volume-title":"Spherical harmonics","author":"M\u00fcller C.","unstructured":"M\u00fcller , C. 1966. Spherical harmonics . Springer . M\u00fcller, C. 1966. Spherical harmonics. Springer."},{"key":"e_1_2_2_37_1","unstructured":"Pighin F. and Lewis J. 2006. Performance-driven facial animation. In ACM SIGGRAPH Courses.  Pighin F. and Lewis J. 2006. Performance-driven facial animation. In ACM SIGGRAPH Courses."},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/280814.280825"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383271"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-010-0380-4"},{"key":"e_1_2_2_41_1","volume-title":"Automatic Face and Gesture Recognition Workshops, 213--220","author":"Saragih J. M.","unstructured":"Saragih , J. M. , Lucey , S. , and Cohn , J. F . 2011. Real-time avatar animation from a single image . In Automatic Face and Gesture Recognition Workshops, 213--220 . Saragih, J. M., Lucey, S., and Cohn, J. F. 2011. Real-time avatar animation from a single image. In Automatic Face and Gesture Recognition Workshops, 213--220."},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661229.2661290"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015706.1015736"},{"key":"e_1_2_2_44_1","volume-title":"Proc. ECCV, 796--812","author":"Suwajanakorn S.","unstructured":"Suwajanakorn , S. , Kemelmacher-Shlizerman , I. , and Seitz , S. M . 2014. Total moving face reconstruction . In Proc. ECCV, 796--812 . Suwajanakorn, S., Kemelmacher-Shlizerman, I., and Seitz, S. M. 2014. Total moving face reconstruction. In Proc. ECCV, 796--812."},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/2366145.2366206"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073209"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2004.00800.x"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/1599470.1599472"},{"key":"e_1_2_2_49_1","doi-asserted-by":"crossref","unstructured":"Weise T. Bouaziz S. Li H. and Pauly M. 2011. Realtime performance-based facial animation. 77.  Weise T. Bouaziz S. Li H. and Pauly M. 2011. Realtime performance-based facial animation. 77.","DOI":"10.1145\/1964921.1964972"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/97879.97906"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/1731047.1731055"},{"key":"e_1_2_2_52_1","volume-title":"Proc. CVPR, 535--542","author":"Xiao J.","unstructured":"Xiao , J. , Baker , S. , Matthews , I. , and Kanade , T . 2004. Real-time combined 2D+3D active appearance models . In Proc. CVPR, 535--542 . Xiao, J., Baker, S., Matthews, I., and Kanade, T. 2004. Real-time combined 2D+3D active appearance models. In Proc. CVPR, 535--542."},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015706.1015759"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601165"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2816795.2818056","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2816795.2818056","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T05:48:18Z","timestamp":1750225698000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2816795.2818056"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,11,2]]},"references-count":54,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2015,11,4]]}},"alternative-id":["10.1145\/2816795.2818056"],"URL":"https:\/\/doi.org\/10.1145\/2816795.2818056","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2015,11,2]]},"assertion":[{"value":"2015-11-02","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}