{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,24]],"date-time":"2026-02-24T17:22:25Z","timestamp":1771953745363,"version":"3.50.1"},"reference-count":49,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2022,11,30]],"date-time":"2022-11-30T00:00:00Z","timestamp":1669766400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"RGC General Research Fund","award":["CUHK14201017,CUHK14201918"],"award-info":[{"award-number":["CUHK14201017,CUHK14201918"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2022,12]]},"abstract":"<jats:p>We present an approach to decompose cartoon animation videos into a set of \"sprites\" --- the basic units of digital cartoons that depict the contents and transforms of each animated object. The sprites in real-world cartoons are unique: artists may draw arbitrary sprite animations for expressiveness, where the animated content is often complicated, irregular, and challenging; alternatively, artists may also reduce their workload by tweening and adjusting sprites, or even reuse static sprites, in which case the transformations are relatively regular and simple. Based on these observations, we propose a sprite decomposition framework using Pixel Multilayer Perceptrons (Pixel MLPs) where the estimation of each sprite is conditioned on and guided by all other sprites. In this way, once those relatively regular and simple sprites are resolved, the decomposition of the remaining \"challenging\" sprites can simplified and eased with the guidance of other sprites. We call this method \"sprite-from-sprite\" cartoon decomposition. We study ablative architectures of our framework, and the user study demonstrates that our results are the most preferred ones in 19\/20 cases.<\/jats:p>","DOI":"10.1145\/3550454.3555439","type":"journal-article","created":{"date-parts":[[2022,11,30]],"date-time":"2022-11-30T21:19:07Z","timestamp":1669843147000},"page":"1-12","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Sprite-from-Sprite"],"prefix":"10.1145","volume":"41","author":[{"given":"Lvmin","family":"Zhang","sequence":"first","affiliation":[{"name":"The Chinese University of Hong Kong, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tien-Tsin","family":"Wong","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuxin","family":"Liu","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Hong Kong, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,11,30]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Aljosa Smolic, and Marc Pollefeys.","author":"Aksoy Yagiz","year":"2017","unstructured":"Yagiz Aksoy, Tung Ozan Aydin, Aljosa Smolic, and Marc Pollefeys. 2017. Unmixingbased soft color segmentation for image manipulation. ACM Transactions on Graphics (2017)."},{"key":"e_1_2_1_2_1","doi-asserted-by":"crossref","unstructured":"Thomas Brox and Jitendra Malik. 2010. Object Segmentation by Long Term Analysis of Point Trajectories.","DOI":"10.1007\/978-3-642-15555-0_21"},{"key":"e_1_2_1_3_1","volume-title":"Palette-based Photo Recoloring. ACM Transactions on Graphics","author":"Chang Huiwen","year":"2015","unstructured":"Huiwen Chang, Ohad Fried, Yiming Liu, Stephen DiVerdi, and Adam Finkelstein. 2015. Palette-based Photo Recoloring. ACM Transactions on Graphics (2015)."},{"key":"e_1_2_1_4_1","volume-title":"Towards Segmenting Anything That Moves. In ICCV Workshops.","author":"Dave Achal","year":"2019","unstructured":"Achal Dave, Pavel Tokmakov, and Deva Ramanan. 2019. Towards Segmenting Anything That Moves. In ICCV Workshops."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201326"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00030"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323006"},{"key":"e_1_2_1_8_1","doi-asserted-by":"crossref","unstructured":"Yoni Kasten Dolev Ofri Oliver Wang and Tali Dekel. 2021. Layered Neural Atlases for Consistent Video Editing.","DOI":"10.1145\/3478513.3480546"},{"key":"e_1_2_1_9_1","doi-asserted-by":"crossref","unstructured":"Margret Keuper. 2017. Higher-Order Minimum Cost Lifted Multicuts for Motion Segmentation.","DOI":"10.1109\/ICCV.2017.455"},{"key":"e_1_2_1_10_1","doi-asserted-by":"crossref","unstructured":"Margret Keuper Bjoern Andres and Thomas Brox. 2015. Motion Trajectory Segmentation via Minimum Cost Multicuts.","DOI":"10.1109\/ICCV.2015.374"},{"key":"e_1_2_1_11_1","unstructured":"Yeong Jun Koh and Chang-Su Kim. 2017. Primary Object Segmentation in Videos Based on Region Augmentation and Reduction."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01249-6_27"},{"key":"e_1_2_1_13_1","unstructured":"Fuxin Li Taeyoung Kim Ahmad Humayun David Tsai and James M Rehg. 2013. Video segmentation by tracking many figure-ground segments."},{"key":"e_1_2_1_14_1","volume-title":"Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes. arXiv preprint arXiv:2011.13084","author":"Li Zhengqi","year":"2020","unstructured":"Zhengqi Li, Simon Niklaus, Noah Snavely, and Oliver Wang. 2020. Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes. arXiv preprint arXiv:2011.13084 (2020)."},{"key":"e_1_2_1_15_1","doi-asserted-by":"crossref","unstructured":"T.-Y. Lin P. Goyal R. Girshick K. He and P. Doll\u00e1r. 2017. Focal loss for dense object detection. ICCV (2017).","DOI":"10.1109\/ICCV.2017.324"},{"key":"e_1_2_1_16_1","volume-title":"Stereoscopizing Cel Animations. ACM Transactions on Graphics (SIGGRAPH Asia 2013 issue) 32, 6 (November","author":"Liu Xueting","year":"2013","unstructured":"Xueting Liu, Xiangyu Mao, Xuan Yang, Linling Zhang, and Tien-Tsin Wong. 2013. Stereoscopizing Cel Animations. ACM Transactions on Graphics (SIGGRAPH Asia 2013 issue) 32, 6 (November 2013), 223:1--223:10."},{"key":"e_1_2_1_17_1","unstructured":"Erika Lu Forrester Cole Tali Dekel Weidi Xie Andrew Zisserman David Salesin William T Freeman and Michael Rubinstein. 2020. Layered Neural Rendering for Retiming People in Video. In SIGGRAPH Asia."},{"key":"e_1_2_1_18_1","volume-title":"Omnimatte: Associating Objects and Their Effects in Video. In CVPR.","author":"Lu Erika","year":"2021","unstructured":"Erika Lu, Forrester Cole, Tali Dekel, Andrew Zisserman, William T Freeman, and Michael Rubinstein. 2021. Omnimatte: Associating Objects and Their Effects in Video. In CVPR."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00459"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58452-8_24"},{"key":"e_1_2_1_21_1","doi-asserted-by":"crossref","unstructured":"Tom Monnier Elliot Vincent Jean Ponce and Mathieu Aubry. 2021. Unsupervised Layered Image Decomposition into Object Prototypes. In ICCV.","DOI":"10.1109\/ICCV48922.2021.00852"},{"key":"e_1_2_1_22_1","doi-asserted-by":"crossref","unstructured":"R.M.H. Nguyen S. Cohen B. Price and M. S. Brown. 2017. GroupTheme Recoloring for Multi-Image Color Consistency. Computer Graphics Forum (2017).","DOI":"10.1111\/cgf.13274"},{"key":"e_1_2_1_23_1","doi-asserted-by":"crossref","unstructured":"Peter Ochs and Thomas Brox. 2011. Object segmentation in video: A hierarchical variational approach for turning point trajectories into dense regions.","DOI":"10.1109\/ICCV.2011.6126418"},{"key":"e_1_2_1_24_1","volume-title":"Segmentation of moving objects by long term video analysis. 36, 6","author":"Ochs Peter","year":"2014","unstructured":"Peter Ochs, Jitendra Malik, and Thomas Brox. 2014a. Segmentation of moving objects by long term video analysis. 36, 6 (2014)."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2013.242"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00288"},{"key":"e_1_2_1_27_1","volume-title":"DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Park Jeong Joon","year":"2019","unstructured":"Jeong Joon Park, Peter Florence, Julian Straub, Richard Newcombe, and Steven Lovegrove. 2019. DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_1_28_1","doi-asserted-by":"crossref","unstructured":"F. Perazzi J. Pont-Tuset B. McWilliams L. Van Gool M. Gross and A. Sorkine-Hornung. 2016. A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation.","DOI":"10.1109\/CVPR.2016.85"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/1399504.1360616"},{"key":"e_1_2_1_30_1","unstructured":"Jianbo Shi and J. Malik. 1998. Motion segmentation and tracking using normalized cuts."},{"key":"e_1_2_1_31_1","volume-title":"Implicit neural representations with periodic activation functions. Advances in Neural Information Processing Systems 33","author":"Sitzmann Vincent","year":"2020","unstructured":"Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. 2020. Implicit neural representations with periodic activation functions. Advances in Neural Information Processing Systems 33 (2020)."},{"key":"e_1_2_1_32_1","unstructured":"Dmitriy Smirnov Michael Gharbi Matthew Fisher Vitor Guizilini Alexei A. Efros and Justin Solomon. 2021. MarioNette: Self-Supervised Sprite Learning."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2024676.2024689"},{"key":"e_1_2_1_34_1","volume-title":"Proceedings of Eurographics Workshop on Sketch-Based Interfaces and Modeling. 27--34","author":"S\u00fdkora Daniel","year":"2005","unstructured":"Daniel S\u00fdkora, Jan Buri\u00e1nek, and Ji\u0159\u00ed \u017d\u00e1ra. 2005. Sketching Cartoons by Example. In Proceedings of Eurographics Workshop on Sketch-Based Interfaces and Modeling. 27--34."},{"key":"e_1_2_1_35_1","volume-title":"Decomposing Images into Layers via RGB-space Geometry. ACM Transactions on Graphics","author":"Tan Jianchao","year":"2017","unstructured":"Jianchao Tan, Jyh-Ming Lien, and Yotam Ginglod. 2017. Decomposing Images into Layers via RGB-space Geometry. ACM Transactions on Graphics (2017)."},{"key":"e_1_2_1_36_1","volume-title":"Efficient palette-based decomposition and recoloring of images via RGBXY-space geometry. ACM Transactions on Graphics","author":"Tan Jianchao","year":"2018","unstructured":"Jianchao Tan, Jyh-Ming Lien, and Yotam Ginglod. 2018. Efficient palette-based decomposition and recoloring of images via RGBXY-space geometry. ACM Transactions on Graphics (2018)."},{"key":"e_1_2_1_37_1","volume-title":"Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains. NeurIPS","author":"Tancik Matthew","year":"2020","unstructured":"Matthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T. Barron, and Ren Ng. 2020. Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains. NeurIPS (2020)."},{"key":"e_1_2_1_38_1","volume-title":"RAFT: Recurrent All-Pairs Field Transforms for Optical Flow. arXiv:2003.12039 [cs.CV]","author":"Teed Zachary","year":"2020","unstructured":"Zachary Teed and Jia Deng. 2020. RAFT: Recurrent All-Pairs Field Transforms for Optical Flow. arXiv:2003.12039 [cs.CV]"},{"key":"e_1_2_1_39_1","doi-asserted-by":"crossref","unstructured":"Lance Williams. 1983. Pyramidal Parametrics.","DOI":"10.1145\/800059.801126"},{"key":"e_1_2_1_40_1","unstructured":"Christopher Xie Yu Xiang Zaid Harchaoui and Dieter Fox. 2019. Object Discovery in Videos as Foreground Motion Clustering."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2816795.2818079"},{"key":"e_1_2_1_42_1","doi-asserted-by":"crossref","unstructured":"Charig Yang Hala Lamdouar Erika Lu Andrew Zisserman and Weidi Xie. 2021b. Self-supervised Video Object Segmentation by Motion Grouping.","DOI":"10.1109\/ICCV48922.2021.00709"},{"key":"e_1_2_1_43_1","doi-asserted-by":"crossref","unstructured":"Yanchao Yang Brian Lai and Stefano Soatto. 2021a. DyStaB: Unsupervised Object Segmentation via Dynamic-Static Bootstrapping.","DOI":"10.1109\/CVPR46437.2021.00285"},{"key":"e_1_2_1_44_1","doi-asserted-by":"crossref","unstructured":"Yanchao Yang Antonio Loquercio Davide Scaramuzza and Stefano Soatto. 2019. Un-supervised Moving Object Detection via Contextual Information Separation.","DOI":"10.1109\/CVPR.2019.00097"},{"key":"e_1_2_1_45_1","volume-title":"Deformable Sprites for Unsupervised Video Decomposition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Ye Vickie","year":"2022","unstructured":"Vickie Ye, Zhengqi Li, Richard Tucker, Angjoo Kanazawa, and Noah Snavely. 2022. Deformable Sprites for Unsupervised Video Decomposition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_1_46_1","unstructured":"Hong-Xing Yu Leonidas J. Guibas and Jiajun Wu. 2022. Unsupervised Discovery of Object Radiance Fields."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2017.2671779"},{"key":"e_1_2_1_48_1","volume-title":"Martin","author":"Zhang Song-Hai","year":"2009","unstructured":"Song-Hai Zhang, Tao Chen, Yi-Fei Zhang, Shi-Min Hu, and Ralph R. Martin. 2009. Vectorizing Cartoon Animations. TVCG (2009)."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925872"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3550454.3555439","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3550454.3555439","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:49:11Z","timestamp":1750182551000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3550454.3555439"}},"subtitle":["Cartoon Animation Decomposition with Self-supervised Sprite Estimation"],"short-title":[],"issued":{"date-parts":[[2022,11,30]]},"references-count":49,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["10.1145\/3550454.3555439"],"URL":"https:\/\/doi.org\/10.1145\/3550454.3555439","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,30]]},"assertion":[{"value":"2022-11-30","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}