{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T06:30:23Z","timestamp":1782282623541,"version":"3.54.5"},"reference-count":42,"publisher":"Association for Computing Machinery (ACM)","issue":"3","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2005,7]]},"abstract":"<jats:p>We present a technique for capturing an actor's live-action performance in such a way that the lighting and reflectance of the actor can be designed and modified in postproduction. Our approach is to illuminate the subject with a sequence of time-multiplexed basis lighting conditions, and to record these conditions with a high-speed video camera so that many conditions are recorded in the span of the desired output frame interval. We investigate several lighting bases for representing the sphere of incident illumination using a set of discrete LED light sources, and we estimate and compensate for subject motion using optical flow and image warping based on a set of tracking frames inserted into the lighting basis. To composite the illuminated performance into a new background, we include a time-multiplexed matte within the basis. We also show that the acquired data enables time-varying surface normals, albedo, and ambient occlusion to be estimated, which can be used to transform the actor's reflectance to produce both subtle and stylistic effects.<\/jats:p>","DOI":"10.1145\/1073204.1073258","type":"journal-article","created":{"date-parts":[[2005,11,7]],"date-time":"2005-11-07T16:00:45Z","timestamp":1131379245000},"page":"756-764","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":180,"title":["Performance relighting and reflectance transformation with time-multiplexed illumination"],"prefix":"10.1145","volume":"24","author":[{"given":"Andreas","family":"Wenger","sequence":"first","affiliation":[{"name":"University of Southern California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrew","family":"Gardner","sequence":"additional","affiliation":[{"name":"University of Southern California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chris","family":"Tchou","sequence":"additional","affiliation":[{"name":"University of Southern California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jonas","family":"Unger","sequence":"additional","affiliation":[{"name":"University of Southern California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tim","family":"Hawkins","sequence":"additional","affiliation":[{"name":"University of Southern California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paul","family":"Debevec","sequence":"additional","affiliation":[{"name":"University of Southern California"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2005,7]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Fourth International Conf. on Computer Vision, 231--236","author":"Black M. J.","unstructured":"Black , M. J. , and Anandan , P . 1993. A framework for the robust estimation of optical flow . In Fourth International Conf. on Computer Vision, 231--236 . Black, M. J., and Anandan, P. 1993. A framework for the robust estimation of optical flow. In Fourth International Conf. on Computer Vision, 231--236."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383325"},{"key":"e_1_2_2_3_1","volume-title":"Advanced Renderman: Creating CGI for Motion Pictures","author":"Calahan S.","year":"2000","unstructured":"Calahan , S. 2000 . Advanced Renderman: Creating CGI for Motion Pictures . Morgan Kaufman Publishers , San Francisco , ch. Storytelling through lighting, a computer perspective, 337382. Calahan, S. 2000. Advanced Renderman: Creating CGI for Motion Pictures. Morgan Kaufman Publishers, San Francisco, ch. Storytelling through lighting, a computer perspective, 337382."},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/344779.344855"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/566654.566614"},{"key":"e_1_2_2_6_1","volume-title":"The Technique of Special Effects Cinematography","author":"Fielding R.","unstructured":"Fielding , R. 1985. The Technique of Special Effects Cinematography , 4 th ed. Hastings House , New York . Fielding, R. 1985. The Technique of Special Effects Cinematography, 4th ed. Hastings House, New York.","edition":"4"},{"key":"e_1_2_2_7_1","volume-title":"IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes, 47--54","author":"Georghiades A.","unstructured":"Georghiades . A. , Belhumeur . P. , and Kriegman , D . 1999. Illumination-based image synthesis: Creating novel images of human faces under differing pose and lighting . In IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes, 47--54 . Georghiades. A., Belhumeur. P., and Kriegman, D. 1999. Illumination-based image synthesis: Creating novel images of human faces under differing pose and lighting. In IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes, 47--54."},{"key":"e_1_2_2_8_1","volume-title":"Eurographics Symposium on Rendering: 14th Eurographics Workshop on Rendering, 230--240","author":"Georghiades A. S.","year":"2003","unstructured":"Georghiades . A. S. 2003 . Recovering 3-d shape and reflectance from a small number of photographs . In Eurographics Symposium on Rendering: 14th Eurographics Workshop on Rendering, 230--240 . Georghiades. A. S. 2003. Recovering 3-d shape and reflectance from a small number of photographs. In Eurographics Symposium on Rendering: 14th Eurographics Workshop on Rendering, 230--240."},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/344779.344938"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.1986.276658"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/280814.280822"},{"key":"e_1_2_2_12_1","unstructured":"Haeberli P. 1992. Synthetic lighting for photography. Available at http:\/\/www.sgi.com\/grafica\/synth\/index.html January.  Haeberli P. 1992. Synthetic lighting for photography. Available at http:\/\/www.sgi.com\/grafica\/synth\/index.html January."},{"key":"e_1_2_2_13_1","volume-title":"Proc. IEEE Conf. on Comp. Vision and Patt. Recog., 403--410","author":"Hager G. D.","unstructured":"Hager , G. D. , and Belhumeur , P. N . 1996. Real-time tracking of image regions with changes in geometry and illumination . In Proc. IEEE Conf. on Comp. Vision and Patt. Recog., 403--410 . Hager, G. D., and Belhumeur, P. N. 1996. Real-time tracking of image regions with changes in geometry and illumination. In Proc. IEEE Conf. on Comp. Vision and Patt. Recog., 403--410."},{"key":"e_1_2_2_14_1","volume-title":"Combinatorial Theory","author":"Hall M.","unstructured":"Hall , M. 1998. Combinatorial Theory , 2 nd ed. Wiley , New York . Hall, M. 1998. Combinatorial Theory, 2nd ed. Wiley, New York.","edition":"2"},{"key":"e_1_2_2_15_1","unstructured":"Harwit M. and Sloane N. J. A. 1979. Hadamard transform optics. Academic Press New York.  Harwit M. and Sloane N. J. A. 1979. Hadamard transform optics. Academic Press New York."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.5555\/2383533.2383575"},{"key":"e_1_2_2_17_1","unstructured":"Hewlett-Packard Components Group. 1998. Noise sources in cmos image sensors. Tech. rep. Hewlett-Packard.  Hewlett-Packard Components Group. 1998. Noise sources in cmos image sensors. Tech. rep. Hewlett-Packard."},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/882262.882270"},{"key":"e_1_2_2_19_1","volume-title":"Course Notes for SIGGRAPH 2002 Course 16","author":"Landis H.","year":"2002","unstructured":"Landis , H. , 2002 . Production-ready global illumination . Course Notes for SIGGRAPH 2002 Course 16 , RenderMan in Production. Landis, H., 2002. Production-ready global illumination. Course Notes for SIGGRAPH 2002 Course 16, RenderMan in Production."},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383320"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/258734.258887"},{"key":"e_1_2_2_22_1","doi-asserted-by":"crossref","unstructured":"Marschner S. Guenter B. and Raghupathy S. 2000. Modeling and rendering for realistic facial animation. In Rendering Techniques 2000: 11th Eurographics Workshop on Rendering 231--242.   Marschner S. Guenter B. and Raghupathy S. 2000. Modeling and rendering for realistic facial animation. In Rendering Techniques 2000: 11th Eurographics Workshop on Rendering 231--242.","DOI":"10.1007\/978-3-7091-6303-0_21"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.5555\/2383533.2383573"},{"key":"e_1_2_2_24_1","unstructured":"Miller G. S. and Hoffman C. R. 1984. Illumination and reflection maps: Simulated objects in simulated and real environments. In SIGGRAPH 84 Course Notes for Advanced Computer Graphics Animation.  Miller G. S. and Hoffman C. R. 1984. Illumination and reflection maps: Simulated objects in simulated and real environments. In SIGGRAPH 84 Course Notes for Advanced Computer Graphics Animation."},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/882262.882280"},{"key":"e_1_2_2_26_1","volume-title":"Fifth Eurographics Workshop on Rendering, 359--373","author":"Nimeroff J. S.","unstructured":"Nimeroff , J. S. , Simoncelli , E. , and Dorsey , J . 1994, Efficient re-rendering of naturally illuminated environments . In Fifth Eurographics Workshop on Rendering, 359--373 . Nimeroff, J. S., Simoncelli, E., and Dorsey, J. 1994, Efficient re-rendering of naturally illuminated environments. In Fifth Eurographics Workshop on Rendering, 359--373."},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015706.1015783"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/344779.345073"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/964965.808606"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015706.1015779"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/566654.566600"},{"key":"e_1_2_2_32_1","volume-title":"International Conference on Computer Vision.","author":"Schechner Y. Y.","unstructured":"Schechner , Y. Y. , Nayar , S. K. , and Belhumeur , P . 2003. A theory of multiplexed illumination . In International Conference on Computer Vision. Schechner, Y. Y., Nayar, S. K., and Belhumeur, P. 2003. A theory of multiplexed illumination. In International Conference on Computer Vision."},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/566654.566612"},{"key":"e_1_2_2_34_1","unstructured":"Sloan N. J. A. 1999. A library of hadamard matrices. http:\/\/www.research.att.com\/~njas\/hadamard\/.  Sloan N. J. A. 1999. A library of hadamard matrices. http:\/\/www.research.att.com\/~njas\/hadamard\/."},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/237170.237263"},{"key":"e_1_2_2_36_1","unstructured":"Trumbull D. 2000. Personal communication. January.  Trumbull D. 2000. Personal communication. January."},{"key":"e_1_2_2_37_1","volume-title":"2003 Conference on Computer Vision and Pattern Recognition (CVPR","author":"Wen Z.","year":"2003","unstructured":"Wen , Z. , Liu , Z. , and Huang , T. S . 2003. Face relighting with radiance environment maps . In 2003 Conference on Computer Vision and Pattern Recognition (CVPR 2003 ), 158--165. Wen, Z., Liu, Z., and Huang, T. S. 2003. Face relighting with radiance environment maps. In 2003 Conference on Computer Vision and Pattern Recognition (CVPR 2003), 158--165."},{"key":"e_1_2_2_38_1","volume-title":"Graphics Interface '91","author":"Williams L.","year":"1991","unstructured":"Williams , L. 1991 . Shading in two dimensions . In Graphics Interface '91 , 143--151. Williams, L. 1991. Shading in two dimensions. In Graphics Interface '91, 143--151."},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1117\/12.7972479"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/122718.122734"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015706.1015759"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/311535.311558"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1073204.1073258","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,4]],"date-time":"2023-09-04T14:00:22Z","timestamp":1693836022000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1073204.1073258"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2005,7]]},"references-count":42,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2005,7]]}},"alternative-id":["10.1145\/1073204.1073258"],"URL":"https:\/\/doi.org\/10.1145\/1073204.1073258","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2005,7]]},"assertion":[{"value":"2005-07-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}