{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T02:49:55Z","timestamp":1774925395600,"version":"3.50.1"},"reference-count":53,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2022,7,1]],"date-time":"2022-07-01T00:00:00Z","timestamp":1656633600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2022,7]]},"abstract":"<jats:p>Procedural material graphs are a compact, parameteric, and resolution-independent representation that are a popular choice for material authoring. However, designing procedural materials requires significant expertise and publicly accessible libraries contain only a few thousand such graphs. We present MatFormer, a generative model that can produce a diverse set of high-quality procedural materials with complex spatial patterns and appearance. While procedural materials can be modeled as directed (operation) graphs, they contain arbitrary numbers of heterogeneous nodes with unstructured, often long-range node connections, and functional constraints on node parameters and connections. MatFormer addresses these challenges with a multi-stage transformer-based model that sequentially generates nodes, node parameters, and edges, while ensuring the semantic validity of the graph. In addition to generation, MatFormer can be used for the auto-completion and exploration of partial material graphs. We qualitatively and quantitatively demonstrate that our method outperforms alternative approaches, in both generated graph and material quality.<\/jats:p>","DOI":"10.1145\/3528223.3530173","type":"journal-article","created":{"date-parts":[[2022,7,22]],"date-time":"2022-07-22T21:06:27Z","timestamp":1658523987000},"page":"1-12","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":42,"title":["MatFormer"],"prefix":"10.1145","volume":"41","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7568-2849","authenticated-orcid":false,"given":"Paul","family":"Guerrero","sequence":"first","affiliation":[{"name":"Adobe Research, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3808-6092","authenticated-orcid":false,"given":"Milo\u0161","family":"Ha\u0161an","sequence":"additional","affiliation":[{"name":"Adobe Research"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6030-2348","authenticated-orcid":false,"given":"Kalyan","family":"Sunkavalli","sequence":"additional","affiliation":[{"name":"Adobe Research"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5558-0327","authenticated-orcid":false,"given":"Radom\u00edr","family":"M\u011bch","sequence":"additional","affiliation":[{"name":"Adobe Research"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5985-0921","authenticated-orcid":false,"given":"Tamy","family":"Boubekeur","sequence":"additional","affiliation":[{"name":"Adobe Research, France"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2597-0914","authenticated-orcid":false,"given":"Niloy J.","family":"Mitra","sequence":"additional","affiliation":[{"name":"Adobe Research, UK and University College London, UK"}]}],"member":"320","published-online":{"date-parts":[[2022,7,22]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"crossref","unstructured":"Rameen Abdal Yipeng Qin and Peter Wonka. 2019. Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?. In ICCV.","DOI":"10.1109\/ICCV.2019.00453"},{"key":"e_1_2_2_2_1","doi-asserted-by":"crossref","unstructured":"Rameen Abdal Peihao Zhu Niloy J. Mitra and Peter Wonka. 2021. StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows. In ACM SIGGRAPH.","DOI":"10.1145\/3447648"},{"key":"e_1_2_2_3_1","unstructured":"Adobe. 2019. Substance. www.substance3d.com."},{"key":"e_1_2_2_4_1","unstructured":"Adobe. 2021a. Substance. https:\/\/substance3d.adobe.com\/community-assets."},{"key":"e_1_2_2_5_1","unstructured":"Adobe. 2021b. Substance. https:\/\/substance3d.adobe.com\/assets."},{"key":"e_1_2_2_6_1","volume-title":"Jamie Ryan Kiros, and Geoffrey E Hinton","author":"Ba Jimmy Lei","year":"2016","unstructured":"Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450 (2016)."},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/W14-4012"},{"key":"e_1_2_2_8_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980","author":"Diederik Kingma","year":"2014","unstructured":"Kingma Diederik, Ba Jimmy, et al. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014), 273--297."},{"key":"e_1_2_2_9_1","volume-title":"Yewen Pu, Andrew Spielberg, Adriana Schulz, Daniela Rus, Armando Solar-Lezama, and Wojciech Matusik.","author":"Du Tao","year":"2018","unstructured":"Tao Du, Jeevana Priya Inala, Yewen Pu, Andrew Spielberg, Adriana Schulz, Daniela Rus, Armando Solar-Lezama, and Wojciech Matusik. 2018. InverseCSG: Automatic Conversion of 3D Models to CSG Trees. In ACM SIGGRAPH Asia."},{"key":"e_1_2_2_10_1","volume-title":"Dreamcoder: Bootstrapping inductive program synthesis with wake-sleep library learning. In ACM SIGPLAN.","author":"Ellis Kevin","year":"2021","unstructured":"Kevin Ellis, Catherine Wong, Maxwell Nye, Mathias Sabl\u00e9-Meyer, Lucas Morales, Luke Hewitt, Luc Cary, Armando Solar-Lezama, and Joshua B Tenenbaum. 2021. Dreamcoder: Bootstrapping inductive program synthesis with wake-sleep library learning. In ACM SIGPLAN."},{"key":"e_1_2_2_11_1","doi-asserted-by":"crossref","unstructured":"Bruno Galerne Ares Lagae Sylvain Lefebvre and George Drettakis. 2012. Gabor Noise by Example. In ACM SIGGRAPH.","DOI":"10.1145\/2185520.2335424"},{"key":"e_1_2_2_12_1","volume-title":"A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576","author":"Gatys Leon A","year":"2015","unstructured":"Leon A Gatys, Alexander S Ecker, and Matthias Bethge. 2015a. A neural algorithm of artistic style. arXiv preprint arXiv:1508.06576 (2015)."},{"key":"e_1_2_2_13_1","volume-title":"Texture synthesis and the controlled generation of natural stimuli using convolutional neural networks. CoRR abs\/1505.07376","author":"Gatys Leon A.","year":"2015","unstructured":"Leon A. Gatys, Alexander S. Ecker, and Matthias Bethge. 2015b. Texture synthesis and the controlled generation of natural stimuli using convolutional neural networks. CoRR abs\/1505.07376 (2015)."},{"key":"e_1_2_2_14_1","unstructured":"Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative Adversarial Nets. In NeurIPS."},{"key":"e_1_2_2_15_1","volume-title":"A Bayesian Inference Framework for Procedural Material Parameter Estimation. Computer Graphics Forum 39, 7","author":"Guo Yu","year":"2020","unstructured":"Yu Guo, Milo\u0161 Ha\u0161an, Lingqi Yan, and Shuang Zhao. 2020a. A Bayesian Inference Framework for Procedural Material Parameter Estimation. Computer Graphics Forum 39, 7 (2020)."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417779"},{"key":"e_1_2_2_17_1","unstructured":"Erik H\u00e4rk\u00f6nen Aaron Hertzmann Jaakko Lehtinen and Sylvain Paris. 2020. GANSpace: Discovering Interpretable GAN Controls. In NeurIPS."},{"key":"e_1_2_2_18_1","volume-title":"The elements of statistical learning: data mining, inference, and prediction","author":"Hastie Trevor","unstructured":"Trevor Hastie, Robert Tibshirani, Jerome H Friedman, and Jerome H Friedman. 2009. The elements of statistical learning: data mining, inference, and prediction. Vol. 2. Springer."},{"key":"e_1_2_2_19_1","volume-title":"Learning a Neural 3D Texture Space from 2D Exemplars","author":"Henzler Philipp","unstructured":"Philipp Henzler, Niloy J Mitra, and Tobias Ritschel. 2019. Learning a Neural 3D Texture Space from 2D Exemplars. In IEEE CVPR."},{"key":"e_1_2_2_20_1","unstructured":"Martin Heusel Hubert Ramsauer Thomas Unterthiner Bernhard Nessler and Sepp Hochreiter. 2017. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In NeurIPS."},{"key":"e_1_2_2_21_1","volume-title":"Long short-term memory. Neural computation 9, 8","author":"Hochreiter Sepp","year":"1997","unstructured":"Sepp Hochreiter and J\u00fcrgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735--1780."},{"key":"e_1_2_2_22_1","volume-title":"Filip De Turck, and Pieter Abbeel","author":"Houthooft Rein","year":"2016","unstructured":"Rein Houthooft, Xi Chen, Yan Duan, John Schulman, Filip De Turck, and Pieter Abbeel. 2016. Curiosity-driven Exploration in Deep Reinforcement Learning via Bayesian Neural Networks. CoRR abs\/1605.09674 (2016)."},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356516"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3502431"},{"key":"e_1_2_2_25_1","doi-asserted-by":"crossref","unstructured":"R. Kenny Jones Theresa Barton Xianghao Xu Kai Wang Ellen Jiang Paul Guerrero Niloy J. Mitra and Daniel Ritchie. 2020. ShapeAssembly: Learning to Generate Programs for 3D Shape Structure Synthesis. In ACM SIGGRAPH Asia.","DOI":"10.1145\/3414685.3417812"},{"key":"e_1_2_2_26_1","volume-title":"A Style-Based Generator Architecture for Generative Adversarial Networks","author":"Karras Tero","unstructured":"Tero Karras, Samuli Laine, and Timo Aila. 2019. A Style-Based Generator Architecture for Generative Adversarial Networks. In IEEE CVPR."},{"key":"e_1_2_2_27_1","doi-asserted-by":"crossref","unstructured":"Tero Karras Samuli Laine Miika Aittala Janne Hellsten Jaakko Lehtinen and Timo Aila. 2020. Analyzing and Improving the Image Quality of StyleGAN.","DOI":"10.1109\/CVPR42600.2020.00813"},{"key":"e_1_2_2_28_1","unstructured":"Manoj Kumar Mohammad Babaeizadeh Dumitru Erhan Chelsea Finn Sergey Levine Laurent Dinh and Durk Kingma. 2020. VideoFlow: A Flow-Based Generative Model for Video. In ICML."},{"key":"e_1_2_2_29_1","unstructured":"Laurent Lefebvre and Pierre Poulin. 2000. Analysis and Synthesis of Structural Textures. In Graphics Interface."},{"key":"e_1_2_2_30_1","volume-title":"Guibas","author":"Li Jun","year":"2017","unstructured":"Jun Li, Kai Xu, Siddhartha Chaudhuri, Ersin Yumer, Hao Zhang, and Leonidas J. Guibas. 2017. GRASS: Generative Recursive Autoencoders for Shape Structures. In ACM SIGGRAPH."},{"key":"e_1_2_2_31_1","volume-title":"Synthesizing 3d shapes from silhouette image collections using multi-projection generative adversarial networks","author":"Li Xiao","unstructured":"Xiao Li, Yue Dong, Pieter Peers, and Xin Tong. 2019. Synthesizing 3d shapes from silhouette image collections using multi-projection generative adversarial networks. In IEEE CVPR."},{"key":"e_1_2_2_32_1","unstructured":"Kaichun Mo Paul Guerrero Li Yi Hao Su Peter Wonka Niloy Mitra and Leonidas Guibas. 2019. StructureNet: Hierarchical Graph Networks for 3D Shape Generation. In ACM SIGGRAPH Asia."},{"key":"e_1_2_2_33_1","unstructured":"Charlie Nash Yaroslav Ganin S. M. Ali Eslami and Peter Battaglia. 2020. PolyGen: An Autoregressive Generative Model of 3D Meshes. In ICML."},{"key":"e_1_2_2_34_1","unstructured":"Aaron Van Oord Nal Kalchbrenner and Koray Kavukcuoglu. 2016. Pixel Recurrent Neural Networks. In ICML."},{"key":"e_1_2_2_35_1","unstructured":"Wamiq Para Shariq Bhat Paul Guerrero Tom Kelly Niloy Mitra Leonidas J. Guibas and Peter Wonka. 2021. SketchGen: Generating Constrained CAD Sketches. In NeurIPS."},{"key":"e_1_2_2_36_1","doi-asserted-by":"crossref","unstructured":"Darwyn R. Peachey. 1985. Solid Texturing of Complex Surfaces. In ACM SIGGRAPH.","DOI":"10.1145\/325165.325246"},{"key":"e_1_2_2_37_1","doi-asserted-by":"crossref","unstructured":"Ken Perlin. 1985. An Image Synthesizer. In ACM SIGGRAPH.","DOI":"10.1145\/325165.325247"},{"key":"e_1_2_2_38_1","unstructured":"Alec Radford Jeffrey Wu Rewon Child David Luan Dario Amodei Ilya Sutskever et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1 8 (2019) 9."},{"key":"e_1_2_2_39_1","volume-title":"Sketchformer: Transformer-based Representation for Sketched Structure","author":"Ferraz Ribeiro Leo Sampaio","year":"2020","unstructured":"Leo Sampaio Ferraz Ribeiro, Tu Bui, John Collomosse, and Moacir Ponti. 2020. Sketchformer: Transformer-based Representation for Sketched Structure. In IEEE CVPR."},{"key":"e_1_2_2_40_1","volume-title":"Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation","author":"Richardson Elad","unstructured":"Elad Richardson, Yuval Alaluf, Or Patashnik, Yotam Nitzan, Yaniv Azar, Stav Shapiro, and Daniel Cohen-Or. 2021. Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation. In IEEE CVPR."},{"key":"e_1_2_2_41_1","volume-title":"Carlo Tomasi, and Leonidas J Guibas","author":"Rubner Yossi","year":"1998","unstructured":"Yossi Rubner, Carlo Tomasi, and Leonidas J Guibas. 1998. A metric for distributions with applications to image databases. In IEEE ICCV."},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNN.2008.2005605"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417781"},{"key":"e_1_2_2_44_1","unstructured":"Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. In ICLR."},{"key":"e_1_2_2_45_1","volume-title":"Going deeper with convolutions","author":"Szegedy Christian","unstructured":"Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In IEEE CVPR."},{"key":"e_1_2_2_46_1","volume-title":"\u0141 ukasz Kaiser, and Illia Polosukhin","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, \u0141 ukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In NeurIPS."},{"key":"e_1_2_2_47_1","unstructured":"Oriol Vinyals Meire Fortunato and Navdeep Jaitly. 2015. Pointer Networks. In NeurIPS."},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459818"},{"key":"e_1_2_2_49_1","volume-title":"ACM SIGGRAPH. Steven Worley.","author":"Witkin Andrew","year":"1996","unstructured":"Andrew Witkin and Michael Kass. 1991. Reaction-Diffusion Textures. In ACM SIGGRAPH. Steven Worley. 1996. A Cellular Texture Basis Function. In ACM SIGGRAPH."},{"key":"e_1_2_2_50_1","volume-title":"DeepCAD: A Deep Generative Network for Computer-Aided Design Models","author":"Wu Rundi","unstructured":"Rundi Wu, Chang Xiao, and Changxi Zheng. 2021. DeepCAD: A Deep Generative Network for Computer-Aided Design Models. In IEEE ICCV."},{"key":"e_1_2_2_51_1","volume-title":"LayoutTransformer: Scene Layout Generation With Conceptual and Spatial Diversity","author":"Yang Cheng-Fu","unstructured":"Cheng-Fu Yang, Wan-Cyuan Fan, Fu-En Yang, and Yu-Chiang Frank Wang. 2021. LayoutTransformer: Scene Layout Generation With Conceptual and Spatial Diversity. In IEEE CVPR."},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.142635"},{"key":"e_1_2_2_53_1","unstructured":"Jiapeng Zhu Yujun Shen Deli Zhao and Bolei Zhou. 2020. In-domain GAN Inversion for Real Image Editing. In ECCV."}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3528223.3530173","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3528223.3530173","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:02:49Z","timestamp":1750186969000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3528223.3530173"}},"subtitle":["a generative model for procedural materials"],"short-title":[],"issued":{"date-parts":[[2022,7]]},"references-count":53,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,7]]}},"alternative-id":["10.1145\/3528223.3530173"],"URL":"https:\/\/doi.org\/10.1145\/3528223.3530173","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7]]},"assertion":[{"value":"2022-07-22","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}