{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T18:57:34Z","timestamp":1774637854136,"version":"3.50.1"},"reference-count":60,"publisher":"Association for Computing Machinery (ACM)","issue":"4","funder":[{"DOI":"10.13039\/501100012166","name":"National Key R&D Program of China","doi-asserted-by":"crossref","award":["2023YFC3305600"],"award-info":[{"award-number":["2023YFC3305600"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100004731","name":"Zhejiang Provincial Natural Science Foundation","doi-asserted-by":"crossref","award":["LDT23F02024F02"],"award-info":[{"award-number":["LDT23F02024F02"]}],"id":[{"id":"10.13039\/501100004731","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"NSFC","doi-asserted-by":"crossref","award":["No.61822111"],"award-info":[{"award-number":["No.61822111"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"NSFC","doi-asserted-by":"crossref","award":["62021002"],"award-info":[{"award-number":["62021002"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"THUIBCS"},{"DOI":"10.13039\/501100004147","name":"Tsinghua University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004147","id-type":"DOI","asserted-by":"crossref"}]},{"name":"BLBCI"},{"DOI":"10.13039\/501100003213","name":"Beijing Municipal Education Commission","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100003213","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2025,8,1]]},"abstract":"<jats:p>Existing facial appearance capture methods can reconstruct plausible facial reflectance from smartphone-recorded videos. However, the reconstruction quality is still far behind the ones based on studio recordings. This paper fills the gap by developing a novel daily-used solution with a co-located smartphone and flashlight video capture setting in a dim room. To enhance the quality, our key observation is to solve facial reflectance maps within the data distribution of studio-scanned ones. Specifically, we first learn a diffusion prior over the Light Stage scans and then steer it to produce the reflectance map that best matches the captured images. We propose to train the diffusion prior at the patch level to improve generalization ability and training stability, as current Light Stage datasets are in ultra-high resolution but limited in data size. Tailored to this prior, we propose a patch-level posterior sampling technique to sample seamless full-resolution reflectance maps from this patch-level diffusion model. Experiments demonstrate our method closes the quality gap between low-cost and studio recordings by a large margin, opening the door for everyday users to clone themselves to the digital world.<\/jats:p>","DOI":"10.1145\/3730825","type":"journal-article","created":{"date-parts":[[2025,7,27]],"date-time":"2025-07-27T04:02:41Z","timestamp":1753588961000},"page":"1-16","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Facial Appearance Capture at Home with Patch-Level Reflectance Prior"],"prefix":"10.1145","volume":"44","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2844-5074","authenticated-orcid":false,"given":"Yuxuan","family":"Han","sequence":"first","affiliation":[{"name":"School of Software and BNRist, Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-8083-2948","authenticated-orcid":false,"given":"Junfeng","family":"Lyu","sequence":"additional","affiliation":[{"name":"School of Software and BNRist, Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-8065-2475","authenticated-orcid":false,"given":"Kuan","family":"Sheng","sequence":"additional","affiliation":[{"name":"ShanghaiTech University, Shanghai, China"},{"name":"Deemos Technology Co., Ltd., Shanghai, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-5490-8940","authenticated-orcid":false,"given":"Minghao","family":"Que","sequence":"additional","affiliation":[{"name":"School of Software and BNRist, Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4837-7152","authenticated-orcid":false,"given":"Qixuan","family":"Zhang","sequence":"additional","affiliation":[{"name":"ShanghaiTech University, Shanghai, China"},{"name":"Deemos Technology Co., Ltd., Shanghai, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8807-7787","authenticated-orcid":false,"given":"Lan","family":"Xu","sequence":"additional","affiliation":[{"name":"ShanghaiTech University, Shanghai, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0953-1057","authenticated-orcid":false,"given":"Feng","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Software and BNRist, Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,7,27]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2503385.2503387"},{"key":"e_1_2_2_2_1","doi-asserted-by":"crossref","unstructured":"Oleg Alexander Mike Rogers William Lambeth Matt Chiang and Paul Debevec. 2009. The digital emily project: photoreal facial modeling and animation. In Acm siggraph 2009 courses. 1\u201315.","DOI":"10.1145\/1667239.1667251"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01615"},{"key":"e_1_2_2_4_1","volume-title":"High-Fidelity 3D Digital Human Head Creation from RGB-D Selfies. ACM Transactions on Graphics","author":"Bao Linchao","year":"2021","unstructured":"Linchao Bao, Xiangkai Lin, Yajing Chen, Haoxian Zhang, Sheng Wang, Xuefei Zhe, Di Kang, Haozhi Huang, Xinwei Jiang, Jue Wang, Dong Yu, and Zhengyou Zhang. 2021. High-Fidelity 3D Digital Human Head Creation from RGB-D Selfies. ACM Transactions on Graphics (2021)."},{"key":"e_1_2_2_5_1","volume-title":"MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation. arXiv preprint arXiv:2302.08113","author":"Bar-Tal Omer","year":"2023","unstructured":"Omer Bar-Tal, Lior Yariv, Yaron Lipman, and Tali Dekel. 2023. MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation. arXiv preprint arXiv:2302.08113 (2023)."},{"key":"e_1_2_2_6_1","volume-title":"Flare: Fast learning of animatable and relightable mesh avatars. arXiv preprint arXiv:2310.17519","author":"Bharadwaj Shrisha","year":"2023","unstructured":"Shrisha Bharadwaj, Yufeng Zheng, Otmar Hilliges, Michael J Black, and Victoria Fernandez-Abrevaya. 2023. Flare: Fast learning of animatable and relightable mesh avatars. arXiv preprint arXiv:2310.17519 (2023)."},{"key":"e_1_2_2_7_1","volume-title":"Acm Siggraph","volume":"2012","author":"Burley Brent","year":"2012","unstructured":"Brent Burley and Walt Disney Animation Studios. 2012. Physically-based shading at disney. In Acm Siggraph, Vol. 2012. vol. 2012, 1\u20137."},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766943"},{"key":"e_1_2_2_9_1","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision. 9429\u20139439","author":"Chen Anpei","year":"2019","unstructured":"Anpei Chen, Zhang Chen, Guli Zhang, Kenny Mitchell, and Jingyi Yu. 2019. Photorealistic facial details synthesis from single image. In Proceedings of the IEEE\/CVF International Conference on Computer Vision. 9429\u20139439."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-73027-6_26"},{"key":"e_1_2_2_11_1","volume-title":"Diffusion posterior sampling for general noisy inverse problems. arXiv preprint arXiv:2209.14687","author":"Chung Hyungjin","year":"2022","unstructured":"Hyungjin Chung, Jeongsol Kim, Michael T Mccann, Marc L Klasky, and Jong Chul Ye. 2022. Diffusion posterior sampling for general noisy inverse problems. arXiv preprint arXiv:2209.14687 (2022)."},{"key":"e_1_2_2_12_1","volume-title":"Peyman Milanfar, Alexandros G Dimakis, and Mauricio Delbracio.","author":"Daras Giannis","year":"2024","unstructured":"Giannis Daras, Hyungjin Chung, Chieh-Hsin Lai, Yuki Mitsufuji, Jong Chul Ye, Peyman Milanfar, Alexandros G Dimakis, and Mauricio Delbracio. 2024. A survey on diffusion models for inverse problems. arXiv preprint arXiv:2410.00083 (2024)."},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/344779.344855"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01263"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52733.2024.00174"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3596711.3596771"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2070781.2024163"},{"key":"e_1_2_2_18_1","doi-asserted-by":"crossref","unstructured":"Aurel Gruber Edo Collins Abhimitra Meka Franziska Mueller Kripasindhu Sarkar Sergio Orts-Escolano Luca Prasso Jay Busch Markus Gross and Thabo Beeler. 2024. GANtlitz: Ultra High Resolution Generative Model for Multi-Modal Face Textures. In Computer Graphics Forum.","DOI":"10.1111\/cgf.15039"},{"key":"e_1_2_2_19_1","unstructured":"Yuxuan Han Junfeng Lyu and Feng Xu. 2024. High-Quality Facial Geometry and Appearance Capture at Home. In CVPR."},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.00831"},{"key":"e_1_2_2_21_1","volume-title":"Denoising Diffusion Probabilistic Models. arXiv preprint arxiv:2006.11239","author":"Ho Jonathan","year":"2020","unstructured":"Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising Diffusion Probabilistic Models. arXiv preprint arxiv:2006.11239 (2020)."},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00877"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.632"},{"key":"e_1_2_2_24_1","unstructured":"Wenzel Jakob S\u00e9bastien Speierer Nicolas Roussel Merlin Nimier-David Delio Vicini Tizian Zeltner Baptiste Nicolet Miguel Crespo Vincent Leroy and Ziyi Zhang. 2022. Mitsuba 3 renderer. https:\/\/mitsuba-renderer.org."},{"key":"e_1_2_2_25_1","volume-title":"Mixture of diffusers for scene composition and high resolution image generation. arXiv preprint arXiv:2302.02412","author":"Jim\u00e9nez \u00c1lvaro Barbero","year":"2023","unstructured":"\u00c1lvaro Barbero Jim\u00e9nez. 2023. Mixture of diffusers for scene composition and high resolution image generation. arXiv preprint arXiv:2302.02412 (2023)."},{"key":"e_1_2_2_26_1","volume-title":"Proc. NeurIPS.","author":"Karras Tero","year":"2022","unstructured":"Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. 2022. Elucidating the Design Space of Diffusion-Based Generative Models. In Proc. NeurIPS."},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00813"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12594"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417861"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-20068-7_30"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00084"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2021.3125598"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00347"},{"key":"e_1_2_2_34_1","doi-asserted-by":"crossref","unstructured":"William E Lorensen and Harvey E Cline. 1998. Marching cubes: A high resolution 3D surface construction algorithm. In Seminal graphics: pioneering efforts that shaped the field. 347\u2013353.","DOI":"10.1145\/280811.281026"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3618357"},{"key":"e_1_2_2_36_1","first-page":"10","article-title":"Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination","volume":"2007","author":"Ma Wan-Chun","year":"2007","unstructured":"Wan-Chun Ma, Tim Hawkins, Pieter Peers, Charles-Felix Chabert, Malte Weiss, Paul E Debevec, et al. 2007. Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination. Rendering Techniques 2007, 9 (2007), 10.","journal-title":"Rendering Techniques"},{"key":"e_1_2_2_37_1","doi-asserted-by":"crossref","unstructured":"Ben Mildenhall Pratul P. Srinivasan Matthew Tancik Jonathan T. Barron Ravi Ramamoorthi and Ren Ng. 2020. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In ECCV.","DOI":"10.1007\/978-3-030-58452-8_24"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.00809"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3596711.3596772"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.15063"},{"key":"e_1_2_2_41_1","volume-title":"Computer Graphics Forum","author":"Rainer Gilles","unstructured":"Gilles Rainer, Lewis Bridgeman, and Abhijeet Ghosh. 2023. Neural shading fields for efficient facial inverse rendering. In Computer Graphics Forum, Vol. 42. Wiley Online Library, e14943."},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383317"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383271"},{"key":"e_1_2_2_44_1","doi-asserted-by":"crossref","unstructured":"J\u00e9r\u00e9my Riviere Paulo Gotardo Derek Bradley Abhijeet Ghosh and Thabo Beeler. 2020. Single-shot high-quality facial geometry and skin appearance capture. (2020).","DOI":"10.1145\/3386569.3392464"},{"key":"e_1_2_2_45_1","doi-asserted-by":"crossref","unstructured":"L Rout Y Chen A Kumar C Caramanis S Shakkottai and W Chu. 2024. Beyond First-Order Tweedie: Solving Inverse Problems using Latent Diffusion.","DOI":"10.1109\/CVPR52733.2024.00905"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00506"},{"key":"e_1_2_2_47_1","volume-title":"Denoising Diffusion Implicit Models. arXiv:2010.02502 (October","author":"Song Jiaming","year":"2020","unstructured":"Jiaming Song, Chenlin Meng, and Stefano Ermon. 2020. Denoising Diffusion Implicit Models. arXiv:2010.02502 (October 2020). https:\/\/arxiv.org\/abs\/2010.02502"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/1057432.1057456"},{"key":"e_1_2_2_49_1","unstructured":"Peng Wang Lingjie Liu Yuan Liu Christian Theobalt Taku Komura and Wenping Wang. 2021. NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction. NeurIPS (2021)."},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01992"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3596711.3596747"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/1141911.1141987"},{"key":"e_1_2_2_53_1","volume-title":"ReconFusion: 3D Reconstruction with Diffusion Priors. arXiv","author":"Wu Rundi","year":"2023","unstructured":"Rundi Wu, Ben Mildenhall, Philipp Henzler, Keunhong Park, Ruiqi Gao, Daniel Watson, Pratul P. Srinivasan, Dor Verbin, Jonathan T. Barron, Ben Poole, and Aleksander Holynski. 2023. ReconFusion: 3D Reconstruction with Diffusion Priors. arXiv (2023)."},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201364"},{"key":"e_1_2_2_55_1","first-page":"4805","article-title":"Volume rendering of neural implicit surfaces","volume":"34","author":"Yariv Lior","year":"2021","unstructured":"Lior Yariv, Jiatao Gu, Yoni Kasten, and Yaron Lipman. 2021. Volume rendering of neural implicit surfaces. Advances in Neural Information Processing Systems 34 (2021), 4805\u20134815.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_2_56_1","volume-title":"Human Hair Reconstruction with Strand-Aligned 3D Gaussians. ArXiv (Sep","author":"Zakharov Egor","year":"2024","unstructured":"Egor Zakharov, Vanessa Sklyarova, Michael J Black, Giljoo Nam, Justus Thies, and Otmar Hilliges. 2024. Human Hair Reconstruction with Strand-Aligned 3D Gaussians. ArXiv (Sep 2024)."},{"key":"e_1_2_2_57_1","doi-asserted-by":"crossref","unstructured":"Longwen Zhang Qiwei Qiu Hongyang Lin Qixuan Zhang Cheng Shi Wei Yang Ye Shi Sibei Yang Lan Xu and Jingyi Yu. 2023a. DreamFace: Progressive Generation of Animatable 3D Faces under Text Guidance. arXiv:2304.03117 [cs.GR]","DOI":"10.1145\/3592094"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3592093"},{"key":"e_1_2_2_59_1","doi-asserted-by":"crossref","unstructured":"Richard Zhang Phillip Isola Alexei A Efros Eli Shechtman and Oliver Wang. 2018. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In CVPR.","DOI":"10.1109\/CVPR.2018.00068"},{"key":"e_1_2_2_60_1","unstructured":"Ye Zhu Yu Wu Zhiwei Deng Olga Russakovsky and Yan Yan. 2023. Unseen image synthesis with diffusion models. (2023)."}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3730825","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T18:00:18Z","timestamp":1774634418000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3730825"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,27]]},"references-count":60,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,8,1]]}},"alternative-id":["10.1145\/3730825"],"URL":"https:\/\/doi.org\/10.1145\/3730825","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,27]]},"assertion":[{"value":"2025-01-23","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-03-29","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-07-27","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}