{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T02:32:51Z","timestamp":1760149971290,"version":"build-2065373602"},"reference-count":39,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2023,9,17]],"date-time":"2023-09-17T00:00:00Z","timestamp":1694908800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62272019"],"award-info":[{"award-number":["62272019"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Realistic fluid models play an important role in computer graphics applications. However, efficiently reconstructing volumetric fluid flows from monocular videos remains challenging. In this work, we present a novel approach for reconstructing 3D flows from monocular inputs through a physics-based differentiable renderer coupled with joint density and velocity estimation. Our primary contributions include the proposed efficient differentiable rendering framework and improved coupled density and velocity estimation strategy. Rather than relying on automatic differentiation, we derive the differential form of the radiance transfer equation under single scattering. This allows the direct computation of the radiance gradient with respect to density, yielding higher efficiency compared to prior works. To improve temporal coherence in the reconstructed flows, subsequent fluid densities are estimated via a coupled strategy that enables smooth and realistic fluid motions suitable for applications that require high efficiency. Experiments on synthetic and real-world data demonstrated our method\u2019s capacity to reconstruct plausible volumetric flows with smooth dynamics efficiently. Comparisons to prior work on fluid motion reconstruction from monocular video revealed over 50\u2013170x speedups across multiple resolutions.<\/jats:p>","DOI":"10.3390\/e25091348","type":"journal-article","created":{"date-parts":[[2023,9,17]],"date-time":"2023-09-17T06:12:21Z","timestamp":1694931141000},"page":"1348","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Physics-Based Differentiable Rendering for Efficient and Plausible Fluid Modeling from Monocular Video"],"prefix":"10.3390","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4930-9386","authenticated-orcid":false,"given":"Yunchi","family":"Cen","sequence":"first","affiliation":[{"name":"School of Computer Science and Engineering, Beihang University, Beijing 100191, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3006-7412","authenticated-orcid":false,"given":"Qifan","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Beihang University, Beijing 100191, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6351-2538","authenticated-orcid":false,"given":"Xiaohui","family":"Liang","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, Beihang University, Beijing 100191, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,9,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Ihrke, I., and Magnor, M. (2004, January 27\u201329). Image-based tomographic reconstruction of flames. Proceedings of the 2004 ACM SIGGRAPH\/Eurographics symposium on Computer Animation, Grenoble, France.","DOI":"10.1145\/1028523.1028572"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2766958","article-title":"Fluid volume modeling from sparse multi-view images by appearance transfer","volume":"34","author":"Okabe","year":"2015","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3355089.3356545","article-title":"ScalarFlow: A large-scale volumetric data set of real-world scalar transport flows for computer animation and machine learning","volume":"38","author":"Eckert","year":"2019","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1111\/cgf.13511","article-title":"Coupled Fluid Density and Motion from Single Views","volume":"37","author":"Eckert","year":"2018","journal-title":"Comput. Graph. Forum"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Franz, E., Solenthaler, B., and Thuerey, N. (2021, January 20\u201325). Global Transport for Fluid Reconstruction with Learned Self-Supervision. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00168"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Kak, A.C., and Slaney, M. (2001). Principles of Computerized Tomographic Imaging, SIAM.","DOI":"10.1137\/1.9780898719277"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1016\/j.gmod.2006.08.001","article-title":"Adaptive grid optical tomography","volume":"68","author":"Ihrke","year":"2006","journal-title":"Graph. Model."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1409060.1409085","article-title":"Time-resolved 3d capture of non-stationary gas flows","volume":"27","author":"Atcheson","year":"2008","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"933","DOI":"10.1007\/s00348-006-0212-z","article-title":"Tomographic particle image velocimetry","volume":"41","author":"Elsinga","year":"2006","journal-title":"Exp. Fluids"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"125403","DOI":"10.1088\/0957-0233\/21\/12\/125403","article-title":"Three-dimensional synthetic aperture particle image velocimetry","volume":"21","author":"Belden","year":"2010","journal-title":"Meas. Sci. Technol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3072959.3073662","article-title":"Rainbow particle imaging velocimetry for dense 3D fluid velocity imaging","volume":"36","author":"Xiong","year":"2017","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Xiong, J., Fu, Q., Idoughi, R., and Heidrich, W. (2018, January 18\u201322). Reconfigurable rainbow PIV for 3D flow measurement. Proceedings of the 2018 IEEE International Conference on Computational Photography (ICCP), Salt Lake City, UT, USA.","DOI":"10.1109\/ICCPHOT.2018.8368475"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1007\/s00348-018-2660-7","article-title":"Single-camera 3D PTV using particle intensities and structured light","volume":"60","author":"Aljedaani","year":"2019","journal-title":"Exp. Fluids"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"115201","DOI":"10.1088\/0957-0233\/26\/11\/115201","article-title":"Volumetric particle image velocimetry with a single plenoptic camera","volume":"26","author":"Fahringer","year":"2015","journal-title":"Meas. Sci. Technol."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Tan, Z.P., and Thurow, B.S. (2019, January 7\u201311). Time-resolved 3D flow-measurement with a single plenoptic-camera. Proceedings of the AIAA Scitech 2019 Forum, San Diego, CA, USA.","DOI":"10.2514\/6.2019-0267"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1007\/s00348-007-0331-1","article-title":"The background oriented schlieren technique: Sensitivity, accuracy, resolution and application to a three-dimensional density field","volume":"43","author":"Goldhahn","year":"2007","journal-title":"Exp. Fluids"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"467","DOI":"10.1007\/s00348-008-0572-7","article-title":"An evaluation of optical flow algorithms for background oriented schlieren imaging","volume":"46","author":"Atcheson","year":"2009","journal-title":"Exp. Fluids"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1109\/34.990137","article-title":"Dense estimation of fluid flows","volume":"24","author":"Corpetti","year":"2002","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Herlin, I., B\u00e9r\u00e9ziat, D., Mercier, N., and Zhuk, S. (2012, January 7\u201313). Divergence-free motion estimation. Proceedings of the European Conference on Computer Vision, Florence, Italy.","DOI":"10.1007\/978-3-642-33765-9_2"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"2283","DOI":"10.1137\/060660709","article-title":"Simultaneous higher-order optical flow estimation and decomposition","volume":"29","author":"Yuan","year":"2007","journal-title":"SIAM J. Sci. Comput."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"755","DOI":"10.1088\/0957-0233\/18\/3\/027","article-title":"Variational estimation of experimental fluid flows with physics-based spatio-temporal regularization","volume":"18","author":"Ruhnau","year":"2007","journal-title":"Meas. Sci. Technol."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Loper, M.M., and Black, M.J. (2014, January 6\u201312). OpenDR: An approximate differentiable renderer. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10584-0_11"},{"key":"ref_23","first-page":"9609","article-title":"Learning to predict 3d objects with an interpolation-based differentiable renderer","volume":"32","author":"Chen","year":"2019","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Genova, K., Cole, F., Maschinot, A., Sarna, A., Vlasic, D., and Freeman, W.T. (2018, January 18\u201323). Unsupervised training for 3d morphable model regression. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00874"},{"key":"ref_25","unstructured":"Henderson, P., and Ferrari, V. (2018). Learning to generate and reconstruct 3d meshes with only 2d supervision. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Kato, H., Ushiku, Y., and Harada, T. (2018, January 18\u201323). Neural 3d mesh renderer. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00411"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Liu, S., Chen, W., Li, T., and Li, H. (2019). Soft rasterizer: Differentiable rendering for unsupervised single-view mesh reconstruction. arXiv.","DOI":"10.1109\/ICCV.2019.00780"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"222","DOI":"10.1145\/3272127.3275109","article-title":"Differentiable monte carlo ray tracing through edge sampling","volume":"37","author":"Li","year":"2018","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_29","first-page":"146","article-title":"Radiative backpropagation: An adjoint method for lightning-fast differentiable rendering","volume":"39","author":"Speierer","year":"2020","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1145\/3450626.3459804","article-title":"Path replay backpropagation: Differentiating light paths using constant memory and linear time","volume":"40","author":"Vicini","year":"2021","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"562","DOI":"10.1109\/TVCG.2021.3114769","article-title":"Differentiable direct volume rendering","volume":"28","author":"Weiss","year":"2021","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"615","DOI":"10.1080\/00029890.1973.11993339","article-title":"Differentiation under the integral sign","volume":"80","author":"Flanders","year":"1973","journal-title":"Am. Math. Mon."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Leal, L.G. (2007). Advanced Transport Phenomena: Fluid Mechanics and Convective Transport Processes, Cambridge University Press.","DOI":"10.1017\/CBO9780511800245"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Wedel, A., and Cremers, D. (2011). Stereoscopic Scene Flow for 3D Motion Analysis, Springer.","DOI":"10.1007\/978-0-85729-965-9"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1007\/s10851-010-0251-1","article-title":"A first-order primal-dual algorithm for convex problems with applications to imaging","volume":"40","author":"Chambolle","year":"2011","journal-title":"J. Math. Imaging Vis."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1145\/3355089.3356506","article-title":"Taichi: A language for high-performance computation on spatially sparse data structures","volume":"38","author":"Hu","year":"2019","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_37","unstructured":"Hu, Y., Anderson, L., Li, T.M., Sun, Q., Carr, N., Ragan-Kelley, J., and Durand, F. (2020). DiffTaichi: Differentiable Programming for Physical Simulation. arXiv."},{"key":"ref_38","unstructured":"Pfaff, T., and Thuerey, N. (2022, December 10). Mantaflow. Available online: http:\/\/mantaflow.com."},{"key":"ref_39","unstructured":"Jakob, W., Speierer, S., Roussel, N., Nimier-David, M., Vicini, D., Zeltner, T., Nicolet, B., Crespo, M., Leroy, V., and Zhang, Z. (2023, February 05). Mitsuba 3 Renderer. Available online: https:\/\/mitsuba-renderer.org."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/9\/1348\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:52:28Z","timestamp":1760129548000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/9\/1348"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,17]]},"references-count":39,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2023,9]]}},"alternative-id":["e25091348"],"URL":"https:\/\/doi.org\/10.3390\/e25091348","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2023,9,17]]}}}