{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T09:25:32Z","timestamp":1780392332548,"version":"3.54.1"},"reference-count":66,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2018,4,30]],"date-time":"2018-04-30T00:00:00Z","timestamp":1525046400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001659","name":"German Research Foundation","doi-asserted-by":"crossref","award":["GRK-1773"],"award-info":[{"award-number":["GRK-1773"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"crossref"}]},{"name":"ERC Starting","award":["335545 CapReal"],"award-info":[{"award-number":["335545 CapReal"]}]},{"name":"Google Faculty Award"},{"name":"TUM-IAS Rudolf M\u00f6\u00dfbauer Fellowship"},{"name":"Max Planck Center for Visual Computing and Communications"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2018,4,30]]},"abstract":"<jats:p>\n            We propose\n            <jats:italic>FaceVR<\/jats:italic>\n            , a novel image-based method that enables video teleconferencing in VR based on self-reenactment. State-of-the-art face tracking methods in the VR context are focused on the animation of rigged 3D avatars (Li et al. 2015; Olszewski et al. 2016). Although they achieve good tracking performance, the results look cartoonish and not real. In contrast to these model-based approaches, FaceVR enables VR teleconferencing using an image-based technique that results in nearly photo-realistic outputs. The key component of FaceVR is a robust algorithm to perform real-time facial motion capture of an actor who is wearing a head-mounted display (HMD), as well as a new data-driven approach for eye tracking from monocular videos. Based on reenactment of a prerecorded stereo video of the person without the HMD, FaceVR incorporates photo-realistic re-rendering in real time, thus allowing artificial modifications of face and eye appearances. For instance, we can alter facial expressions or change gaze directions in the prerecorded target video. In a live setup, we apply these newly introduced algorithmic components.\n          <\/jats:p>","DOI":"10.1145\/3182644","type":"journal-article","created":{"date-parts":[[2018,6,29]],"date-time":"2018-06-29T15:13:34Z","timestamp":1530285214000},"page":"1-15","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":99,"title":["FaceVR"],"prefix":"10.1145","volume":"37","author":[{"given":"Justus","family":"Thies","sequence":"first","affiliation":[{"name":"Technical University Munich, Garching, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Zollh\u00f6fer","sequence":"additional","affiliation":[{"name":"Stanford University, Stanford, United States of America"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marc","family":"Stamminger","sequence":"additional","affiliation":[{"name":"University of Erlangen-Nuremberg, Erlangen, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christian","family":"Theobalt","sequence":"additional","affiliation":[{"name":"Max-Planck-Institute for Informatics, Saarbruecken, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matthias","family":"Nie\u00dfner","sequence":"additional","affiliation":[{"name":"Technical University Munich, Garching, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2018,6,29]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/1667239.1667251"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964970"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766924"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1111\/1467-8659.t01-1-00712"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/311535.311556"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/965400.965469"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2461976"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/258734.258880"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766943"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601204"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2013.249"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925873"},{"key":"e_1_2_2_13_1","volume-title":"Torr","author":"Criminisi Antonio","year":"2003","unstructured":"Antonio Criminisi , Jamie Shotton , Andrew Blake , and Philip H. S . Torr . 2003 . Gaze manipulation for one-to-one teleconferencing. In Proceedings of ICCV. Antonio Criminisi, Jamie Shotton, Andrew Blake, and Philip H. S. Torr. 2003. Gaze manipulation for one-to-one teleconferencing. In Proceedings of ICCV."},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2070781.2024164"},{"key":"e_1_2_2_15_1","volume-title":"Opt: A domain specific language for non-linear least squares optimization in graphics and imaging. arXiv:1604.06525.","author":"DeVito Zachary","year":"2016","unstructured":"Zachary DeVito , Michael Mara , Michael Zollh\u00f6fer , Gilbert Bernstein , Jonathan Ragan-Kelley , Christian Theobalt , Pat Hanrahan , Matthew Fisher , and Matthias Nie\u00dfner . 2016 . Opt: A domain specific language for non-linear least squares optimization in graphics and imaging. arXiv:1604.06525. Zachary DeVito, Michael Mara, Michael Zollh\u00f6fer, Gilbert Bernstein, Jonathan Ragan-Kelley, Christian Theobalt, Pat Hanrahan, Matthew Fisher, and Matthias Nie\u00dfner. 2016. Opt: A domain specific language for non-linear least squares optimization in graphics and imaging. arXiv:1604.06525."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1143844.1143880"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3084363.3085083"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2638549"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.537"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12552"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508380"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2890493"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298776"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964969"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766974"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.2197\/ipsjjip.22.401"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925871"},{"key":"e_1_2_2_28_1","volume-title":"Seitz","author":"Kemelmacher-Shlizerman Ira","year":"2010","unstructured":"Ira Kemelmacher-Shlizerman , Aditya Sankar , Eli Shechtman , and Steven M . Seitz . 2010 . Being John Malkovich. In Proceedings of ECCV. 341--353. Ira Kemelmacher-Shlizerman, Aditya Sankar, Eli Shechtman, and Steven M. Seitz. 2010. Being John Malkovich. In Proceedings of ECCV. 341--353."},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12594"},{"key":"e_1_2_2_30_1","volume-title":"Proceedings of CVPR. 4667--4675","author":"Kononenko D.","unstructured":"D. Kononenko and V. Lempitsky . 2015. Learning to look up: Realtime monocular gaze correction using machine learning . In Proceedings of CVPR. 4667--4675 . D. Kononenko and V. Lempitsky. 2015. Learning to look up: Realtime monocular gaze correction using machine learning. In Proceedings of CVPR. 4667--4675."},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2366145.2366193"},{"key":"e_1_2_2_32_1","unstructured":"Pupil Labs. 2016. Home Page. Retrieved April 4 2018 from https:\/\/pupil-labs.com\/pupil\/.  Pupil Labs. 2016. Home Page. Retrieved April 4 2018 from https:\/\/pupil-labs.com\/pupil\/."},{"key":"e_1_2_2_33_1","volume-title":"Proceedings of EUROGRAPHICS STAR Reports. 199--218","author":"Lewis J. P.","year":"2014","unstructured":"J. P. Lewis , Ken Anjyo , Taehyun Rhee , Mengjie Zhang , Fred Pighin , and Zhigang Deng . 2014 . Practice and theory of blendshape facial models . In Proceedings of EUROGRAPHICS STAR Reports. 199--218 . J. P. Lewis, Ken Anjyo, Taehyun Rhee, Mengjie Zhang, Fred Pighin, and Zhigang Deng. 2014. Practice and theory of blendshape facial models. In Proceedings of EUROGRAPHICS STAR Reports. 199--218."},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766939"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2462019"},{"key":"e_1_2_2_36_1","volume-title":"Proceedings of CVPR. 57--64","author":"Li Kai","year":"2012","unstructured":"Kai Li , Feng Xu , Jue Wang , Qionghai Dai , and Yebin Liu . 2012 . A data-driven approach for facial expression synthesis in video . In Proceedings of CVPR. 57--64 . Kai Li, Feng Xu, Jue Wang, Qionghai Dai, and Yebin Liu. 2012. A data-driven approach for facial expression synthesis in video. In Proceedings of CVPR. 57--64."},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2980252"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2009.23"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/1201775.882269"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/280814.280825"},{"key":"e_1_2_2_41_1","volume-title":"Proceedings of ACM SIGGRAPH Courses.","author":"Pighin F.","unstructured":"F. Pighin and J. P. Lewis . 2006. Performance-driven facial animation . In Proceedings of ACM SIGGRAPH Courses. F. Pighin and J. P. Lewis. 2006. Performance-driven facial animation. In Proceedings of ACM SIGGRAPH Courses."},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383271"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46484-8_15"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-010-0380-4"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661229.2661290"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2017.2734428"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.235"},{"key":"e_1_2_2_48_1","volume-title":"Seitz","author":"Suwajanakorn Supasorn","year":"2014","unstructured":"Supasorn Suwajanakorn , Ira Kemelmacher-Shlizerman , and Steven M . Seitz . 2014 . Total moving face reconstruction. In Proceedings of ECCV. 796--812. Supasorn Suwajanakorn, Ira Kemelmacher-Shlizerman, and Steven M. Seitz. 2014. Total moving face reconstruction. In Proceedings of ECCV. 796--812."},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.450"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073640"},{"key":"e_1_2_2_51_1","volume-title":"Proceedings of ICASSP. IEEE","author":"Taylor Sarah L.","unstructured":"Sarah L. Taylor , Barry-John Theobald , and Iain A. Matthews . 2015. A mouth full of words: Visually consistent acoustic redubbing . In Proceedings of ICASSP. IEEE , Los Alamitos, CA, 4904--4908. Sarah L. Taylor, Barry-John Theobald, and Iain A. Matthews. 2015. A mouth full of words: Visually consistent acoustic redubbing. In Proceedings of ICASSP. IEEE, Los Alamitos, CA, 4904--4908."},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964971"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/2816795.2818056"},{"key":"e_1_2_2_54_1","volume-title":"Proceedings of CVPR.","author":"Thies Justus","unstructured":"Justus Thies , M. Zollh\u00f6fer , M. Stamminger , C. Theobalt , and M. Nie\u00dfner . 2016. Face2Face: Real-time face capture and reenactment of RGB videos . In Proceedings of CVPR. Justus Thies, M. Zollh\u00f6fer, M. Stamminger, C. Theobalt, and M. Nie\u00dfner. 2016. Face2Face: Real-time face capture and reenactment of RGB videos. In Proceedings of CVPR."},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/2366145.2366206"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073209"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925947"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/1409060.1409071"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964972"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/1599470.1599472"},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/97879.97906"},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661229.2661232"},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015706.1015759"},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7299081"},{"key":"e_1_2_2_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766887"},{"key":"e_1_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601165"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3182644","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3182644","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T01:39:13Z","timestamp":1750210753000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3182644"}},"subtitle":["Real-Time Gaze-Aware Facial Reenactment in Virtual Reality"],"short-title":[],"issued":{"date-parts":[[2018,4,30]]},"references-count":66,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2018,4,30]]}},"alternative-id":["10.1145\/3182644"],"URL":"https:\/\/doi.org\/10.1145\/3182644","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,4,30]]},"assertion":[{"value":"2017-09-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-02-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-06-29","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}