{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,9]],"date-time":"2026-02-09T22:36:31Z","timestamp":1770676591644,"version":"3.49.0"},"reference-count":64,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2020,11,27]],"date-time":"2020-11-27T00:00:00Z","timestamp":1606435200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Google Faculty Award"},{"name":"ERC","award":["335373"],"award-info":[{"award-number":["335373"]}]},{"name":"Adobe"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2020,12,31]]},"abstract":"<jats:p>\n            Patterns, which are collections of elements arranged in regular or near-regular arrangements, are an important graphic art form and widely used due to their elegant simplicity and aesthetic appeal. When a pattern is encoded as a flat image without the underlying structure, manually editing the pattern is tedious and challenging as one has to both preserve the individual element shapes and their original relative arrangements. State-of-the-art deep learning frameworks that operate at the pixel level are unsuitable for manipulating such patterns. Specifically, these methods can easily disturb the shapes of the individual elements or their arrangement, and thus fail to preserve the latent structures of the input patterns. We present a novel differentiable\n            <jats:italic>compositing<\/jats:italic>\n            operator using pattern elements and use it to discover structures, in the form of a layered representation of graphical objects, directly from raw pattern images. This operator allows us to adapt current deep learning based image methods to effectively handle patterns. We evaluate our method on a range of patterns and demonstrate superiority in the context of pattern manipulations when compared against state-of-the-art pixel- or point-based alternatives.\n          <\/jats:p>","DOI":"10.1145\/3414685.3417830","type":"journal-article","created":{"date-parts":[[2020,11,27]],"date-time":"2020-11-27T21:51:05Z","timestamp":1606513865000},"page":"1-15","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":21,"title":["Discovering pattern structure using differentiable compositing"],"prefix":"10.1145","volume":"39","author":[{"given":"Pradyumna","family":"Reddy","sequence":"first","affiliation":[{"name":"University College London"}]},{"given":"Paul","family":"Guerrero","sequence":"additional","affiliation":[{"name":"Adobe Research"}]},{"given":"Matt","family":"Fisher","sequence":"additional","affiliation":[{"name":"Adobe Research"}]},{"given":"Wilmot","family":"Li","sequence":"additional","affiliation":[{"name":"Adobe Research"}]},{"given":"Niloy J.","family":"Mitra","sequence":"additional","affiliation":[{"name":"Adobe Research and University College London"}]}],"member":"320","published-online":{"date-parts":[[2020,11,27]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Proceedings of the 35th International Conference on Machine Learning","volume":"80","author":"Achlioptas Panos","year":"2018","unstructured":"Panos Achlioptas , Olga Diamanti , Ioannis Mitliagkas , and Leonidas Guibas . 2018 . Learning Representations and Generative Models for 3D Point Clouds . In Proceedings of the 35th International Conference on Machine Learning , Vol. 80 . 40--49. Panos Achlioptas, Olga Diamanti, Ioannis Mitliagkas, and Leonidas Guibas. 2018. Learning Representations and Generative Models for 3D Point Clouds. In Proceedings of the 35th International Conference on Machine Learning, Vol. 80. 40--49."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46466-4_10"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1576246.1531330"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1117\/12.421129"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01182"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778820"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298813"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383296"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.265"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925950"},{"key":"e_1_2_2_11_1","volume-title":"Toward Convolutional Blind Denoising of Real Photographs. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Guo Shi","year":"2019","unstructured":"Shi Guo , Zifei Yan , Kai Zhang , Wangmeng Zuo , and Lei Zhang . 2019 . Toward Convolutional Blind Denoising of Real Photographs. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Shi Guo, Zifei Yan, Kai Zhang, Wangmeng Zuo, and Lei Zhang. 2019. Toward Convolutional Blind Denoising of Real Photographs. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_2_12_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3279--3286","author":"Han Xufeng","year":"2015","unstructured":"Xufeng Han , Thomas Leung , Yangqing Jia , Rahul Sukthankar , and Alexander C Berg . 2015 . Matchnet: Unifying feature and metric learning for patch-based matching . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3279--3286 . Xufeng Han, Thomas Leung, Yangqing Jia, Rahul Sukthankar, and Alexander C Berg. 2015. Matchnet: Unifying feature and metric learning for patch-based matching. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3279--3286."},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2019.01.005"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2487228.2487233"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2005.184"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383295"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1572614.1572623"},{"key":"e_1_2_2_18_1","volume-title":"Statistical analysis and modelling of spatial point patterns","unstructured":"Janine. Illian. 2008. Statistical analysis and modelling of spatial point patterns . John Wiley . 534 pages. Janine. Illian. 2008. Statistical analysis and modelling of spatial point patterns. John Wiley. 534 pages."},{"key":"e_1_2_2_19_1","unstructured":"Max Jaderberg Karen Simonyan Andrew Zisserman etal 2015. Spatial transformer networks. In Advances in neural information processing systems. 2017--2025.  Max Jaderberg Karen Simonyan Andrew Zisserman et al. 2015. Spatial transformer networks. In Advances in neural information processing systems. 2017--2025."},{"key":"e_1_2_2_20_1","unstructured":"Y. Jing Y. Yang Z. Feng J. Ye Y. Yu and M. Song. 2019. Neural Style Transfer: A Review. IEEE Transactions on Visualization and Computer Graphics (2019) 1--1.  Y. Jing Y. Yang Z. Feng J. Ye Y. Yu and M. Song. 2019. Neural Style Transfer: A Review. IEEE Transactions on Visualization and Computer Graphics (2019) 1--1."},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00453"},{"key":"e_1_2_2_22_1","volume-title":"Kingma and Jimmy Ba","author":"Diederik","year":"2014","unstructured":"Diederik P. Kingma and Jimmy Ba . 2014 . Adam : A Method for Stochastic Optimization . arXiv:cs.LG\/1412.6980 Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. arXiv:cs.LG\/1412.6980"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.302"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/882262.882264"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12152"},{"key":"e_1_2_2_26_1","volume-title":"Proceedings of the 35th International Conference on Machine Learning","volume":"80","author":"Lehtinen Jaakko","year":"2018","unstructured":"Jaakko Lehtinen , Jacob Munkberg , Jon Hasselgren , Samuli Laine , Tero Karras , Miika Aittala , and Timo Aila . 2018 . Noise2Noise: Learning Image Restoration without Clean Data . In Proceedings of the 35th International Conference on Machine Learning , Vol. 80 . 2965--2974. Jaakko Lehtinen, Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras, Miika Aittala, and Timo Aila. 2018. Noise2Noise: Learning Image Restoration without Clean Data. In Proceedings of the 35th International Conference on Machine Learning, Vol. 80. 2965--2974."},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356562"},{"key":"e_1_2_2_28_1","volume-title":"Proc. Vision Interface. 120--123","author":"Lewis J.P.","year":"1995","unstructured":"J.P. Lewis . 1995 . Fast Template Matching . In Proc. Vision Interface. 120--123 . J.P. Lewis. 1995. Fast Template Matching. In Proc. Vision Interface. 120--123."},{"key":"e_1_2_2_29_1","volume-title":"Point Cloud GAN. (10","author":"Li Chun-Liang","year":"2018","unstructured":"Chun-Liang Li , Manzil Zaheer , Yang Zhang , Barnabas Poczos , and Ruslan Salakhutdinov . 2018. Point Cloud GAN. (10 2018 ). http:\/\/arxiv.org\/abs\/1810.05795 Chun-Liang Li, Manzil Zaheer, Yang Zhang, Barnabas Poczos, and Ruslan Salakhutdinov. 2018. Point Cloud GAN. (10 2018). http:\/\/arxiv.org\/abs\/1810.05795"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/501786.501787"},{"key":"e_1_2_2_31_1","first-page":"221","article-title":"Paparazzi: surface editing by way of multi-view image processing","volume":"37","author":"Derek Liu Hsueh-Ti","year":"2018","unstructured":"Hsueh-Ti Derek Liu , Michael Tao , and Alec Jacobson . 2018 . Paparazzi: surface editing by way of multi-view image processing . ACM Trans. Graph. 37 , 6 (2018), 221 -- 221 . Hsueh-Ti Derek Liu, Michael Tao, and Alec Jacobson. 2018. Paparazzi: surface editing by way of multi-view image processing. ACM Trans. Graph. 37, 6 (2018), 221--1.","journal-title":"ACM Trans. Graph."},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00780"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356510"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.614"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964957"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2008.919362"},{"key":"e_1_2_2_37_1","volume-title":"Local Layering. In ACM SIGGRAPH 2009 Papers. Article 84","author":"McCann James","year":"2009","unstructured":"James McCann and Nancy Pollard . 2009 . Local Layering. In ACM SIGGRAPH 2009 Papers. Article 84 , 7 pages. James McCann and Nancy Pollard. 2009. Local Layering. In ACM SIGGRAPH 2009 Papers. Article 84, 7 pages."},{"key":"e_1_2_2_38_1","volume-title":"Asian Conference on Computer Vision. Springer, 638--653","author":"Melekhov Iaroslav","year":"2016","unstructured":"Iaroslav Melekhov , Juho Kannala , and Esa Rahtu . 2016 . Image patch matching using convolutional descriptors with euclidean distance . In Asian Conference on Computer Vision. Springer, 638--653 . Iaroslav Melekhov, Juho Kannala, and Esa Rahtu. 2016. Image patch matching using convolutional descriptors with euclidean distance. In Asian Conference on Computer Vision. Springer, 638--653."},{"key":"e_1_2_2_39_1","volume-title":"Neural Importance Sampling. (8","author":"M\u00fcller Thomas","year":"2018","unstructured":"Thomas M\u00fcller , Brian McWilliams , Fabrice Rousselle , Markus Gross , and Jan Nov\u00e1k . 2018. Neural Importance Sampling. (8 2018 ). http:\/\/arxiv.org\/abs\/1808.03856 Thomas M\u00fcller, Brian McWilliams, Fabrice Rousselle, Markus Gross, and Jan Nov\u00e1k. 2018. Neural Importance Sampling. (8 2018). http:\/\/arxiv.org\/abs\/1808.03856"},{"key":"e_1_2_2_40_1","volume-title":"Luigi Di Stefano, and Wai-Kuen Cham","author":"Ouyang Wanli","year":"2011","unstructured":"Wanli Ouyang , Federico Tombari , Stefano Mattoccia , Luigi Di Stefano, and Wai-Kuen Cham . 2011 . Performance evaluation of full search equivalent pattern matching algorithms. IEEE transactions on pattern analysis and machine intelligence 34, 1 (2011), 127--143. Wanli Ouyang, Federico Tombari, Stefano Mattoccia, Luigi Di Stefano, and Wai-Kuen Cham. 2011. Performance evaluation of full search equivalent pattern matching algorithms. IEEE transactions on pattern analysis and machine intelligence 34, 1 (2011), 127--143."},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2366145.2366189"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.12"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/1618452.1618453"},{"key":"e_1_2_2_44_1","volume-title":"ACM transactions on graphics (TOG) 23, 3","author":"Rother Carsten","year":"2004","unstructured":"Carsten Rother , Vladimir Kolmogorov , and Andrew Blake . 2004. \" GrabCut\" interactive foreground extraction using iterated graph cuts. ACM transactions on graphics (TOG) 23, 3 ( 2004 ), 309--314. Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. 2004. \" GrabCut\" interactive foreground extraction using iterated graph cuts. ACM transactions on graphics (TOG) 23, 3 (2004), 309--314."},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13111"},{"key":"e_1_2_2_46_1","volume-title":"Computer Graphics Forum","author":"Roveri Riccardo","unstructured":"Riccardo Roveri , A Cengiz \u00d6ztireli , Sebastian Martin , Barbara Solenthaler , and Markus Gross . 2015. Example based repetitive structure synthesis . In Computer Graphics Forum , Vol. 34 . Wiley Online Library , 39--52. Riccardo Roveri, A Cengiz \u00d6ztireli, Sebastian Martin, Barbara Solenthaler, and Markus Gross. 2015. Example based repetitive structure synthesis. In Computer Graphics Forum, Vol. 34. Wiley Online Library, 39--52."},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00467"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00459"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.22"},{"key":"e_1_2_2_50_1","volume-title":"Very Deep Convolutional Networks for Large-Scale Image Recognition. In International Conference on Learning Representations.","author":"Simonyan Karen","year":"2015","unstructured":"Karen Simonyan and Andrew Zisserman . 2015 . Very Deep Convolutional Networks for Large-Scale Image Recognition. In International Conference on Learning Representations. Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. In International Conference on Learning Representations."},{"key":"e_1_2_2_51_1","volume-title":"Computer Graphics Forum","author":"\u0160t'ava Ondrej","unstructured":"Ondrej \u0160t'ava , Bedrich Bene\u0161 , Radomir M\u011bch , Daniel G Aliaga , and Peter Kri\u0161tof . 2010. Inverse procedural modeling by automatic generation of L-systems . In Computer Graphics Forum , Vol. 29 . Wiley Online Library , 665--674. Ondrej \u0160t'ava, Bedrich Bene\u0161, Radomir M\u011bch, Daniel G Aliaga, and Peter Kri\u0161tof. 2010. Inverse procedural modeling by automatic generation of L-systems. In Computer Graphics Forum, Vol. 29. Wiley Online Library, 665--674."},{"key":"e_1_2_2_52_1","volume-title":"Sarma","author":"Sun Yongbin","year":"2018","unstructured":"Yongbin Sun , Yue Wang , Ziwei Liu , Joshua E. Siegel , and Sanjay E . Sarma . 2018 . Point-Grow: Autoregressively Learned Point Cloud Generation with Self-Attention . (10 2018). http:\/\/arxiv.org\/abs\/1810.05591 Yongbin Sun, Yue Wang, Ziwei Liu, Joshua E. Siegel, and Sanjay E. Sarma. 2018. Point-Grow: Autoregressively Learned Point Cloud Generation with Self-Attention. (10 2018). http:\/\/arxiv.org\/abs\/1810.05591"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.144"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/1944846.1944851"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-48881-3_8"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.5244\/C.30.145"},{"key":"e_1_2_2_57_1","volume-title":"Computer Graphics Forum","author":"Tu Peihan","unstructured":"Peihan Tu , Dani Lischinski , and Hui Huang . 2019. Point Pattern Synthesis via Irregular Convolution . In Computer Graphics Forum , Vol. 38 . Wiley Online Library , 109--122. Peihan Tu, Dani Lischinski, and Hui Huang. 2019. Point Pattern Synthesis via Irregular Convolution. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 109--122."},{"key":"e_1_2_2_58_1","volume-title":"Texture Networks: Feed-forward Synthesis of Textures and Stylized Images. (3","author":"Ulyanov Dmitry","year":"2016","unstructured":"Dmitry Ulyanov , Vadim Lebedev , Andrea Vedaldi , and Victor Lempitsky . 2016 . Texture Networks: Feed-forward Synthesis of Textures and Stylized Images. (3 2016). http:\/\/arxiv.org\/abs\/1603.03417 Dmitry Ulyanov, Vadim Lebedev, Andrea Vedaldi, and Victor Lempitsky. 2016. Texture Networks: Feed-forward Synthesis of Textures and Stylized Images. (3 2016). http:\/\/arxiv.org\/abs\/1603.03417"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2008.2004615"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3123266.3123411"},{"key":"e_1_2_2_61_1","volume-title":"Generative Image Inpainting With Contextual Attention. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Yu Jiahui","unstructured":"Jiahui Yu , Zhe Lin , Jimei Yang , Xiaohui Shen , Xin Lu , and Thomas S. Huang . 2018 . Generative Image Inpainting With Contextual Attention. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S. Huang. 2018. Generative Image Inpainting With Contextual Attention. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_2_62_1","volume-title":"Free-Form Image Inpainting With Gated Convolution. In The IEEE International Conference on Computer Vision (ICCV).","author":"Yu Jiahui","unstructured":"Jiahui Yu , Zhe Lin , Jimei Yang , Xiaohui Shen , Xin Lu , and Thomas S. Huang . 2019 . Free-Form Image Inpainting With Gated Convolution. In The IEEE International Conference on Computer Vision (ICCV). Jiahui Yu, Zhe Lin, Jimei Yang, Xiaohui Shen, Xin Lu, and Thomas S. Huang. 2019. Free-Form Image Inpainting With Gated Convolution. In The IEEE International Conference on Computer Vision (ICCV)."},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185572"},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201286"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3414685.3417830","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3414685.3417830","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:03:15Z","timestamp":1750197795000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3414685.3417830"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,27]]},"references-count":64,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2020,12,31]]}},"alternative-id":["10.1145\/3414685.3417830"],"URL":"https:\/\/doi.org\/10.1145\/3414685.3417830","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,11,27]]},"assertion":[{"value":"2020-11-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}