{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,6]],"date-time":"2026-07-06T12:29:48Z","timestamp":1783340988548,"version":"3.54.6"},"reference-count":39,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2018,7,30]],"date-time":"2018-07-30T00:00:00Z","timestamp":1532908800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2018,8,31]]},"abstract":"<jats:p>We introduce a deep appearance model for rendering the human face. Inspired by Active Appearance Models, we develop a data-driven rendering pipeline that learns a joint representation of facial geometry and appearance from a multiview capture setup. Vertex positions and view-specific textures are modeled using a deep variational autoencoder that captures complex nonlinear effects while producing a smooth and compact latent representation. View-specific texture enables the modeling of view-dependent effects such as specularity. In addition, it can also correct for imperfect geometry stemming from biased or low resolution estimates. This is a significant departure from the traditional graphics pipeline, which requires highly accurate geometry as well as all elements of the shading model to achieve realism through physically-inspired light transport. Acquiring such a high level of accuracy is difficult in practice, especially for complex and intricate parts of the face, such as eyelashes and the oral cavity. These are handled naturally by our approach, which does not rely on precise estimates of geometry. Instead, the shading model accommodates deficiencies in geometry though the flexibility afforded by the neural network employed. At inference time, we condition the decoding network on the viewpoint of the camera in order to generate the appropriate texture for rendering. The resulting system can be implemented simply using existing rendering engines through dynamic textures with flat lighting. This representation, together with a novel unsupervised technique for mapping images to facial states, results in a system that is naturally suited to real-time interactive settings such as Virtual Reality (VR).<\/jats:p>","DOI":"10.1145\/3197517.3201401","type":"journal-article","created":{"date-parts":[[2018,7,31]],"date-time":"2018-07-31T15:56:23Z","timestamp":1533052583000},"page":"1-13","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":247,"title":["Deep appearance models for face rendering"],"prefix":"10.1145","volume":"37","author":[{"given":"Stephen","family":"Lombardi","sequence":"first","affiliation":[{"name":"Facebook Reality Labs"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jason","family":"Saragih","sequence":"additional","affiliation":[{"name":"Facebook Reality Labs"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tomas","family":"Simon","sequence":"additional","affiliation":[{"name":"Facebook Reality Labs"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yaser","family":"Sheikh","sequence":"additional","affiliation":[{"name":"Facebook Reality Labs"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2018,7,30]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/311535.311556"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0021928"},{"key":"e_1_2_2_3_1","volume-title":"Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks. (07","author":"Bousmalis Konstantinos","year":"2017"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925873"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2915926.2915936"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cviu.2015.08.011"},{"key":"e_1_2_2_7_1","volume-title":"Variational Lossy Autoencoder. CoRR abs\/1611.02731","author":"Chen Xi","year":"2016"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.927467"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/300776.300778"},{"key":"e_1_2_2_10_1","volume-title":"Proceedings of the 3rd. International Conference on Face & Gesture Recognition (FG '98)","author":"Edwards G. J."},{"key":"e_1_2_2_11_1","volume-title":"The Face of Man: Expressions of Universal Emotions in a New Guinea Village","author":"Ekman P."},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/237170.237200"},{"key":"e_1_2_2_13_1","volume-title":"Deep Feature Consistent Variational Autoencoder. In 2017 IEEE Winter Conference on Applications of Computer Vision (WACV). 1133--1141","author":"Hou X."},{"key":"e_1_2_2_14_1","volume-title":"Glass","author":"Hsu Wei-Ning","year":"2017"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130800.31310887"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766974"},{"key":"e_1_2_2_17_1","volume-title":"ICML (JMLR Workshop and Conference Proceedings), Francis R. Bach and David M. Blei (Eds.)","volume":"37","author":"Ioffe Sergey","year":"2015"},{"key":"e_1_2_2_18_1","volume-title":"Proceedings 2000 International Conference on Image Processing","volume":"2","author":"Kang Sing Bing"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.241"},{"key":"e_1_2_2_20_1","volume-title":"Jung Kwon Lee, and Jiwon Kim","author":"Kim Taeksoo","year":"2017"},{"key":"e_1_2_2_21_1","volume-title":"Proceedings of the 3rd International Conference on Learning Representations abs\/1412","author":"Diederik","year":"2014"},{"key":"e_1_2_2_22_1","volume-title":"Proceedings of the 2nd International Conference on Learning Representations.","author":"Diederik"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12594"},{"key":"e_1_2_2_24_1","volume-title":"Morphable Models of Faces","author":"Knothe Reinhard"},{"key":"e_1_2_2_25_1","volume-title":"Proceedings of the 28th International Conference on Neural Information Processing Systems (NIPS'15)","author":"Kulkarni Tejas D."},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3099564.3099581"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2010.41"},{"key":"e_1_2_2_28_1","volume-title":"Proc. Eurographics State of The Art Report.","author":"Lewis John P.","year":"2014"},{"key":"e_1_2_2_29_1","unstructured":"Ming-Yu Liu Thomas Breuel and Jan Kautz. 2017. Unsupervised Image-to-image Translation Networks. In NIPS.  Ming-Yu Liu Thomas Breuel and Jan Kautz. 2017. Unsupervised Image-to-image Translation Networks. In NIPS."},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1023\/B:VISI.0000029666.37597.d3"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2980252"},{"key":"e_1_2_2_32_1","volume-title":"Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks. CoRR abs\/1511.06434","author":"Radford Alec","year":"2015"},{"key":"e_1_2_2_33_1","volume-title":"Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks. In Advances in Neural Information Processing Systems 29","author":"Salimans Tim","year":"2016"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.220"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.5555\/839277.840029"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3182644"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-37431-9_50"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.75"},{"key":"e_1_2_2_39_1","volume-title":"Unpaired Image-to-image Translation using Cycle-Consistent Adversarial Networks. (December","author":"Zhu Jun-Yan","year":"2017"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3197517.3201401","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3197517.3201401","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T02:07:00Z","timestamp":1750212420000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3197517.3201401"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,7,30]]},"references-count":39,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2018,8,31]]}},"alternative-id":["10.1145\/3197517.3201401"],"URL":"https:\/\/doi.org\/10.1145\/3197517.3201401","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,7,30]]},"assertion":[{"value":"2018-07-30","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}