{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T06:11:44Z","timestamp":1784268704896,"version":"3.55.0"},"reference-count":45,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2021,12,1]],"date-time":"2021-12-01T00:00:00Z","timestamp":1638316800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Samsung Research Funding & Incubation Center of Samsung Electronics","award":["SRFC-IT2001-04"],"award-info":[{"award-number":["SRFC-IT2001-04"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:p>\n            Recent differentiable rendering techniques have become key tools to tackle many inverse problems in graphics and vision. Existing models, however, assume steady-state light transport, i.e., infinite speed of light. While this is a safe assumption for many applications, recent advances in ultrafast imaging leverage the wealth of information that can be extracted from the exact time of flight of light. In this context, physically-based transient rendering allows to efficiently simulate and analyze light transport considering that the speed of light is indeed finite. In this paper, we introduce a novel differentiable transient rendering framework, to help bring the potential of differentiable approaches into the transient regime. To differentiate the transient path integral we need to take into account that scattering events at path vertices are no longer independent; instead, tracking the time of flight of light requires treating such scattering events at path vertices jointly as a multidimensional, evolving manifold. We thus turn to the generalized transport theorem, and introduce a novel\n            <jats:italic>correlated importance<\/jats:italic>\n            term, which links the time-integrated contribution of a path to its light throughput, and allows us to handle discontinuities in the light and sensor functions. Last, we present results in several challenging scenarios where the time of flight of light plays an important role such as optimizing indices of refraction, non-line-of-sight tracking with nonplanar relay walls, and non-line-of-sight tracking around two corners.\n          <\/jats:p>","DOI":"10.1145\/3478513.3480498","type":"journal-article","created":{"date-parts":[[2021,12,10]],"date-time":"2021-12-10T18:29:20Z","timestamp":1639160960000},"page":"1-11","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["Differentiable transient rendering"],"prefix":"10.1145","volume":"40","author":[{"given":"Shinyoung","family":"Yi","sequence":"first","affiliation":[{"name":"KAIST, South Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Donggun","family":"Kim","sequence":"additional","affiliation":[{"name":"KAIST, South Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kiseok","family":"Choi","sequence":"additional","affiliation":[{"name":"KAIST, South Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Adrian","family":"Jarabo","sequence":"additional","affiliation":[{"name":"Universidad de Zaragoza, SPAIN"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Diego","family":"Gutierrez","sequence":"additional","affiliation":[{"name":"Universidad de Zaragoza, SPAIN"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Min H.","family":"Kim","sequence":"additional","affiliation":[{"name":"KAIST, South Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,12,10]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2016.2567379"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2557605"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417833"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0022112005006695"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417825"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46487-9_42"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2008.01152.x"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2461945"},{"key":"e_1_2_2_9_1","volume-title":"Imaging in scattering media using correlation image sensors and sparse convolutional coding. Opt","author":"Heide Felix","year":"2014","unstructured":"Felix Heide , Lei Xiao , Andreas Kolb , Matthias B Hullin , and Wolfgang Heidrich . 2014. Imaging in scattering media using correlation image sensors and sparse convolutional coding. Opt . Express 22, 21 ( 2014 ). Felix Heide, Lei Xiao, Andreas Kolb, Matthias B Hullin, and Wolfgang Heidrich. 2014. Imaging in scattering media using correlation image sensors and sparse convolutional coding. Opt. Express 22, 21 (2014)."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3368314"},{"key":"e_1_2_2_11_1","volume-title":"Femto-photography: Visualizing light in motion. Master's thesis. Universidad de Zaragoza.","author":"Jarabo Adrian","year":"2012","unstructured":"Adrian Jarabo . 2012 . Femto-photography: Visualizing light in motion. Master's thesis. Universidad de Zaragoza. Adrian Jarabo. 2012. Femto-photography: Visualizing light in motion. Master's thesis. Universidad de Zaragoza."},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13314"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661229.2661251"},{"key":"e_1_2_2_14_1","volume-title":"Recent Advances in Transient Imaging: A Computer Graphics and Vision Perspective. Visual Informatics 1, 1","author":"Jarabo Adrian","year":"2017","unstructured":"Adrian Jarabo , Belen Masia , Julio Marco , and Diego Gutierrez . 2017. Recent Advances in Transient Imaging: A Computer Graphics and Vision Perspective. Visual Informatics 1, 1 ( 2017 ). Adrian Jarabo, Belen Masia, Julio Marco, and Diego Gutierrez. 2017. Recent Advances in Transient Imaging: A Computer Graphics and Vision Perspective. Visual Informatics 1, 1 (2017)."},{"key":"e_1_2_2_15_1","volume-title":"Differentiable rendering: A survey. arXiv preprint arXiv:2006.12057","author":"Kato Hiroharu","year":"2020","unstructured":"Hiroharu Kato , Deniz Beker , Mihai Morariu , Takahiro Ando , Toru Matsuoka , Wadim Kehl , and Adrien Gaidon . 2020. Differentiable rendering: A survey. arXiv preprint arXiv:2006.12057 ( 2020 ). Hiroharu Kato, Deniz Beker, Mihai Morariu, Takahiro Ando, Toru Matsuoka, Wadim Kehl, and Adrien Gaidon. 2020. Differentiable rendering: A survey. arXiv preprint arXiv:2006.12057 (2020)."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00411"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2504435.2504439"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417861"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275109"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00780"},{"key":"e_1_2_2_21_1","volume-title":"Ji Hyun Nam, Syed Azer Reza, Toan Huu Le, Adrian Jarabo, Diego Gutierrez, and Andreas Velten.","author":"Liu Xiaochun","year":"2019","unstructured":"Xiaochun Liu , Ib\u00f3n Guill\u00e9n , Marco La Manna , Ji Hyun Nam, Syed Azer Reza, Toan Huu Le, Adrian Jarabo, Diego Gutierrez, and Andreas Velten. 2019 a. Non-Line-of-Sight Imaging using Phasor Fields Virtual Wave Optics. Nature ( 2019). Xiaochun Liu, Ib\u00f3n Guill\u00e9n, Marco La Manna, Ji Hyun Nam, Syed Azer Reza, Toan Huu Le, Adrian Jarabo, Diego Gutierrez, and Andreas Velten. 2019a. Non-Line-of-Sight Imaging using Phasor Fields Virtual Wave Optics. Nature (2019)."},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10584-0_11"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356510"},{"key":"e_1_2_2_24_1","volume-title":"Progressive Transient Photon Beams. Computer Graphics Forum","author":"Marco Julio","year":"2019","unstructured":"Julio Marco , Ib\u00f3n Guill\u00e9n , Wojciech Jarosz , Diego Gutierrez , and Adrian Jarabo . 2019. Progressive Transient Photon Beams. Computer Graphics Forum ( 2019 ). Julio Marco, Ib\u00f3n Guill\u00e9n, Wojciech Jarosz, Diego Gutierrez, and Adrian Jarabo. 2019. Progressive Transient Photon Beams. Computer Graphics Forum (2019)."},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00244"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392406"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356498"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601103"},{"key":"e_1_2_2_29_1","volume-title":"Confocal non-line-of-sight imaging based on the light-cone transform. Nature 555, 7696","author":"O'Toole Matthew","year":"2018","unstructured":"Matthew O'Toole , David B Lindell , and Gordon Wetzstein . 2018. Confocal non-line-of-sight imaging based on the light-cone transform. Nature 555, 7696 ( 2018 ), 338. Matthew O'Toole, David B Lindell, and Gordon Wetzstein. 2018. Confocal non-line-of-sight imaging based on the light-cone transform. Nature 555, 7696 (2018), 338."},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cag.2019.07.009"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323016"},{"key":"e_1_2_2_33_1","volume-title":"Locating and classifying fluorescent tags behind turbid layers using time-resolved inversion. Nature communications 6, 1","author":"Satat Guy","year":"2015","unstructured":"Guy Satat , Barmak Heshmat , Christopher Barsi , Dan Raviv , Ou Chen , Moungi G Bawendi , and Ramesh Raskar . 2015. Locating and classifying fluorescent tags behind turbid layers using time-resolved inversion. Nature communications 6, 1 ( 2015 ), 1--8. Guy Satat, Barmak Heshmat, Christopher Barsi, Dan Raviv, Ou Chen, Moungi G Bawendi, and Ramesh Raskar. 2015. Locating and classifying fluorescent tags behind turbid layers using time-resolved inversion. Nature communications 6, 1 (2015), 1--8."},{"key":"e_1_2_2_34_1","volume-title":"Ultrafast Bandgap Photonics","volume":"9835","author":"Satat Guy","year":"2016","unstructured":"Guy Satat , Barmak Heshmat , Nikhil Naik , Albert Redo-Sanchez , and Ramesh Raskar . 2016 . Advances in ultrafast optics and imaging applications . In Ultrafast Bandgap Photonics , Vol. 9835 . International Society for Optics and Photonics, 98350Q. Guy Satat, Barmak Heshmat, Nikhil Naik, Albert Redo-Sanchez, and Ramesh Raskar. 2016. Advances in ultrafast optics and imaging applications. In Ultrafast Bandgap Photonics, Vol. 9835. International Society for Optics and Photonics, 98350Q."},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1142\/S0218202514500067"},{"key":"e_1_2_2_36_1","volume-title":"Technical Report UCSC-SOE-08-26. School of Engineering","author":"Smith Adam","unstructured":"Adam Smith , James Skorupski , and James Davis . 2008. Transient Rendering . Technical Report UCSC-SOE-08-26. School of Engineering , University of California , Santa Cruz . Adam Smith, James Skorupski, and James Davis. 2008. Transient Rendering. Technical Report UCSC-SOE-08-26. School of Engineering, University of California, Santa Cruz."},{"key":"e_1_2_2_37_1","volume-title":"Beyond Volumetric Albedo-A Surface Optimization Framework for Non-Line-Of-Sight Imaging. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1545--1555","author":"Tsai Chia-Yin","year":"2019","unstructured":"Chia-Yin Tsai , Aswin C Sankaranarayanan , and Ioannis Gkioulekas . 2019 . Beyond Volumetric Albedo-A Surface Optimization Framework for Non-Line-Of-Sight Imaging. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1545--1555 . Chia-Yin Tsai, Aswin C Sankaranarayanan, and Ioannis Gkioulekas. 2019. Beyond Volumetric Albedo-A Surface Optimization Framework for Non-Line-Of-Sight Imaging. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1545--1555."},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.5555\/927297"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2461928"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459804"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-013-0668-2"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1364\/OL.43.001299"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392383"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356522"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459782"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3388769.3407454"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3478513.3480498","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3478513.3480498","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:11:48Z","timestamp":1750191108000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3478513.3480498"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12]]},"references-count":45,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["10.1145\/3478513.3480498"],"URL":"https:\/\/doi.org\/10.1145\/3478513.3480498","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12]]},"assertion":[{"value":"2021-12-10","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}