{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T17:13:43Z","timestamp":1777655623672,"version":"3.51.4"},"reference-count":87,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2023,7,26]],"date-time":"2023-07-26T00:00:00Z","timestamp":1690329600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2023,8]]},"abstract":"<jats:p>Node-graph-based procedural materials are vital to 3D content creation within the computer graphics industry. Leveraging the expressive representation of procedural materials, artists can effortlessly generate diverse appearances by altering the graph structure or node parameters. However, manually reproducing a specific appearance is a challenging task that demands extensive domain knowledge and labor. Previous research has sought to automate this process by converting artist-created material graphs into differentiable programs and optimizing node parameters against a photographed material appearance using gradient descent. These methods involve implementing differentiable filter nodes [Shi et al. 2020] and training differentiable neural proxies for generator nodes to optimize continuous and discrete node parameters [Hu et al. 2022a] jointly. Nevertheless, Neural Proxies exhibits critical limitations, such as long training times, inaccuracies, fixed resolutions, and confined parameter ranges, which hinder their scalability towards the broad spectrum of production-grade material graphs. These constraints fundamentally stem from the absence of faithful and efficient implementations of generic noise and pattern generator nodes, both differentiable and non-differentiable. Such deficiency prevents the direct optimization of continuous and discrete generator node parameters without relying on surrogate models.<\/jats:p>\n          <jats:p>\n            We present\n            <jats:italic>Diffmat v2<\/jats:italic>\n            , an improved differentiable procedural material library, along with a fully-automated, end-to-end procedural material capture framework that combines gradient-based optimization and gradient-free parameter search to match existing production-grade procedural materials against user-taken flash photos. Diffmat v2 expands the range of differentiable material graph nodes in Diffmat [Shi et al. 2020] by adding generic noise\/pattern generator nodes and user-customizable per-pixel filter nodes. This allows for the complete translation and optimization of procedural materials across various categories without the need for external proprietary tools or pre-cached noise patterns. Consequently, our method can capture a considerably broader array of materials, encompassing those with highly regular or stochastic geometries. We demonstrate that our end-to-end approach yields a closer match to the target than MATch [Shi et al. 2020] and Neural Proxies [Hu et al. 2022a] when starting from initially unmatched continuous and discrete parameters.\n          <\/jats:p>","DOI":"10.1145\/3592132","type":"journal-article","created":{"date-parts":[[2023,7,26]],"date-time":"2023-07-26T15:47:45Z","timestamp":1690386465000},"page":"1-15","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["End-to-end Procedural Material Capture with Proxy-Free Mixed-Integer Optimization"],"prefix":"10.1145","volume":"42","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9271-0055","authenticated-orcid":false,"given":"Beichen","family":"Li","sequence":"first","affiliation":[{"name":"Massachusetts Institute of Technology (MIT), Cambridge, United States of America"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4442-4679","authenticated-orcid":false,"given":"Liang","family":"Shi","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology (MIT), Cambridge, United States of America"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0212-5643","authenticated-orcid":false,"given":"Wojciech","family":"Matusik","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology (MIT), Cambridge, United States of America"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,7,26]]},"reference":[{"key":"e_1_2_2_1_1","unstructured":"Substance 3D Designer Adobe. 2022a. Adobe. https:\/\/substance3d.adobe.com\/documentation\/sddoc\/substance-3d-designer-102400008.html."},{"key":"e_1_2_2_2_1","unstructured":"Substance 3D Designer Function Nodes Overview Adobe. 2022b. Adobe. https:\/\/substance3d.adobe.com\/documentation\/sddoc\/function-nodes-overview-102400052.html."},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2461978"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766967"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.2312\/egtp.19881023"},{"key":"e_1_2_2_6_1","doi-asserted-by":"crossref","unstructured":"Louis-Philippe Asselin Denis Laurendeau and Jean-Francois Lalonde. 2020. Deep SVBRDF Estimation on Real Materials.","DOI":"10.1109\/3DV50981.2020.00126"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417833"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1017\/S0962492913000032"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073264"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/300776.300778"},{"key":"e_1_2_2_11_1","unstructured":"Sebastien Deguy Albert Benassi and Universit\u00e9 Blaise Pascal. 2001. A Flexible Noise Model For Designing Maps. http:\/\/citeseerx.ist.psu.edu \u203a viewdoc \u203a summary- http:\/\/citeseerx.ist.psu.edu \u203a viewdoc \u203a summary (2001)."},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201378"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13765"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1111\/1467-8659.t01-1-00600"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661229.2661283"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1833349.1778835"},{"key":"e_1_2_2_17_1","volume-title":"Texturing and Modeling: A Procedural Approach","author":"Ebert David S.","unstructured":"David S. Ebert, F. Kenton Musgrave, Darwyn Peachey, Ken Perlin, and Steven Worley. 2002. Texturing and Modeling: A Procedural Approach (3rd ed.). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.","edition":"3"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1837026.1837068"},{"key":"e_1_2_2_19_1","first-page":"6","article-title":"Computer rendering of stochastic models","volume":"25","author":"Fournier Alain","year":"1982","unstructured":"Alain Fournier, Don Fussell, and Loren Carpenter. 1982. Computer rendering of stochastic models. Commun. ACM 25, 6 (June 1982), 371--384.","journal-title":"Commun. ACM"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185569"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13073"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323042"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/882262.882342"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/964965.808572"},{"key":"e_1_2_2_25_1","volume-title":"Advances in Neural Information Processing Systems","author":"Gatys Leon","unstructured":"Leon Gatys, Alexander S Ecker, and Matthias Bethge. 2015. Texture Synthesis Using Convolutional Neural Networks. In Advances in Neural Information Processing Systems, C Cortes, N Lawrence, D Lee, M Sugiyama, and R Garnett (Eds.), Vol. 28. Curran Associates, Inc."},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2009.01493.x"},{"key":"e_1_2_2_27_1","volume-title":"South Korea) (SIGGRAPH ASIA '10","author":"Ghosh Abhijeet","year":"2010","unstructured":"Abhijeet Ghosh, Tongbo Chen, Pieter Peers, Cyrus A Wilson, and Paul Debevec. 2010. Circularly polarized spherical illumination reflectometry. In ACM SIGGRAPH Asia 2010 papers (Seoul, South Korea) (SIGGRAPH ASIA '10, Article 162). Association for Computing Machinery, New York, NY, USA, 1--12."},{"key":"e_1_2_2_28_1","doi-asserted-by":"crossref","unstructured":"Abhijeet Ghosh Tim Hawkins Pieter Peers Sune Frederiksen and Paul Debevec. 2008. Practical modeling and acquisition of layered facial reflectance.","DOI":"10.1145\/1457515.1409092"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661229.2661249"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360653"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2009.102"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3528223.3530173"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459854"},{"key":"e_1_2_2_34_1","volume-title":"A Bayesian Inference Framework for Procedural Material Parameter Estimation. (Dec","author":"Guo Yu","year":"2019","unstructured":"Yu Guo, Milos Hasan, Lingqi Yan, and Shuang Zhao. 2019. A Bayesian Inference Framework for Procedural Material Parameter Estimation. (Dec. 2019). arXiv:1912.01067 [cs.GR]"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417779"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3233304"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3478513.3480507"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778836"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356516"},{"key":"e_1_2_2_40_1","volume-title":"Node Graph Optimization Using Differentiable Proxies. In ACM SIGGRAPH 2022 Conference Proceedings","author":"Hu Yiwei","year":"2022","unstructured":"Yiwei Hu, Paul Guerrero, Milos Hasan, Holly Rushmeier, and Valentin Deschaintre. 2022a. Node Graph Optimization Using Differentiable Proxies. In ACM SIGGRAPH 2022 Conference Proceedings (Vancouver, BC, Canada) (SIGGRAPH '22, Article 5). Association for Computing Machinery, New York, NY, USA, 1--9."},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3502431"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356506"},{"key":"e_1_2_2_43_1","doi-asserted-by":"crossref","unstructured":"Zhuo Hui Kalyan Sunkavalli Joon-Young Lee Sunil Hadap Jian Wang and Aswin C Sankaranarayanan. 2017. Reflectance Capture Using Univariate Sampling of BRDFs.","DOI":"10.1109\/ICCV.2017.573"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356492"},{"key":"e_1_2_2_45_1","volume-title":"Differentiable Rendering: A Survey. (June","author":"Kato Hiroharu","year":"2020","unstructured":"Hiroharu Kato, Deniz Beker, Mihai Morariu, Takahiro Ando, Toru Matsuoka, Wadim Kehl, and Adrien Gaidon. 2020. Differentiable Rendering: A Survey. (June 2020). arXiv:2006.12057 [cs.CV]"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2010.01827.x"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/1576246.1531360"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cag.2010.05.004"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/1141911.1141949"},{"key":"e_1_2_2_50_1","volume-title":"Mixed integer nonlinear programming","author":"Lee Jon","unstructured":"Jon Lee and Sven Leyffer. 2011. Mixed integer nonlinear programming. Vol. 154. Springer Science & Business Media."},{"key":"e_1_2_2_51_1","volume-title":"Analysis and Synthesis of Structural Textures. Graphics Interface","author":"Lefebvre L","year":"2000","unstructured":"L Lefebvre and Pierre Poulin. 2000. Analysis and Synthesis of Structural Textures. Graphics Interface (2000)."},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/1833349.1778821"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/964965.808605"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.5555\/16564.16594"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/35068.35069"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/74334.74360"},{"key":"e_1_2_2_57_1","doi-asserted-by":"crossref","unstructured":"T M Li M Aittala F Durand and J Lehtinen. 2018a. Differentiable monte carlo ray tracing through edge sampling. ACM Trans. Graph. (2018).","DOI":"10.1145\/3272127.3275109"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073641"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01219-9_5"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275055"},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356510"},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275017"},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2980220"},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356498"},{"key":"e_1_2_2_65_1","volume-title":"ACM SIGGRAPH 2002 Course","author":"Olano M","year":"2002","unstructured":"M Olano, Hartj. C, W Heidrich, B Mark, and K Perlin. 2002. Real-time shading languages. ACM SIGGRAPH 2002 Course (2002)."},{"key":"e_1_2_2_66_1","volume-title":"PyTorch: An imperative style, high-performance deep learning library. Adv. Neural Inf. Process. Syst. 32","author":"Paszke Adam","year":"2019","unstructured":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An imperative style, high-performance deep learning library. Adv. Neural Inf. Process. Syst. 32 (2019)."},{"key":"e_1_2_2_67_1","volume-title":"An image synthesizer. ACM Siggraph Computer Graphics","author":"Perlin K","year":"1985","unstructured":"K Perlin. 1985. An image synthesizer. ACM Siggraph Computer Graphics (1985)."},{"key":"e_1_2_2_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/566570.566636"},{"key":"e_1_2_2_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/344779.344987"},{"key":"e_1_2_2_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964940"},{"key":"e_1_2_2_71_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12719"},{"key":"e_1_2_2_72_1","volume-title":"The Science of Fractal Images, Michael F Barnsley, Robert L Devaney, Benoit B Mandelbrot, Heinz-Otto Peitgen, Dietmar Saupe, Richard F Voss, Heinz-Otto Peitgen","author":"Saupe Dietmar","unstructured":"Dietmar Saupe. 1988. Algorithms for random fractals. In The Science of Fractal Images, Michael F Barnsley, Robert L Devaney, Benoit B Mandelbrot, Heinz-Otto Peitgen, Dietmar Saupe, Richard F Voss, Heinz-Otto Peitgen, and Dietmar Saupe (Eds.). Springer New York, New York, NY, 71--136."},{"key":"e_1_2_2_73_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417781"},{"key":"e_1_2_2_74_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2461944"},{"key":"e_1_2_2_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/127719.122751"},{"key":"e_1_2_2_76_1","volume-title":"The Science of Fractal Images, Michael F Barnsley, Robert L Devaney, Benoit B Mandelbrot, Heinz-Otto Peitgen, Dietmar Saupe, Richard F Voss, Heinz-Otto Peitgen","author":"Voss Richard F","unstructured":"Richard F Voss. 1988. Fractals in nature: From characterization to simulation. In The Science of Fractal Images, Michael F Barnsley, Robert L Devaney, Benoit B Mandelbrot, Heinz-Otto Peitgen, Dietmar Saupe, Richard F Voss, Heinz-Otto Peitgen, and Dietmar Saupe (Eds.). Springer New York, New York, NY, 21--70."},{"key":"e_1_2_2_77_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360640"},{"key":"e_1_2_2_78_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2015.2498617"},{"key":"e_1_2_2_79_1","doi-asserted-by":"publisher","DOI":"10.1145\/311625.312135"},{"key":"e_1_2_2_80_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2982396"},{"key":"e_1_2_2_81_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13560"},{"key":"e_1_2_2_82_1","volume-title":"PhotoScene: Photorealistic Material and Lighting Transfer for Indoor Scenes. (June","author":"Yeh Yu-Ying","year":"2022","unstructured":"Yu-Ying Yeh, Zhengqin Li, Yannick Hold-Geoffroy, Rui Zhu, Zexiang Xu, Milos Hasan, Kalyan Sunkavalli, and Manmohan Chandraker. 2022. PhotoScene: Photorealistic Material and Lighting Transfer for Indoor Scenes. (June 2022)."},{"key":"e_1_2_2_83_1","volume-title":"Henrik Wann Jensen, and Ravi Ramamoorthi","author":"Yu Jiyang","year":"2016","unstructured":"Jiyang Yu, Zexiang Xu, Matteo Mannino, Henrik Wann Jensen, and Ravi Ramamoorthi. 2016. Sparse sampling for image-based SVBRDF acquisition."},{"key":"e_1_2_2_84_1","volume-title":"Path-space differentiable rendering. ACM transactions on","author":"Zhang C","year":"2020","unstructured":"C Zhang, B Miller, K Yan, I Gkioulekas, and others. 2020. Path-space differentiable rendering. ACM transactions on (2020)."},{"key":"e_1_2_2_85_1","doi-asserted-by":"publisher","DOI":"10.1145\/3388769.3407454"},{"key":"e_1_2_2_86_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.142635"},{"key":"e_1_2_2_87_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2980247"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3592132","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3592132","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:37:46Z","timestamp":1750178266000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3592132"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,26]]},"references-count":87,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,8]]}},"alternative-id":["10.1145\/3592132"],"URL":"https:\/\/doi.org\/10.1145\/3592132","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,26]]},"assertion":[{"value":"2023-07-26","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}