{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,18]],"date-time":"2026-07-18T18:26:39Z","timestamp":1784399199511,"version":"3.55.0"},"reference-count":43,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2020,8,12]],"date-time":"2020-08-12T00:00:00Z","timestamp":1597190400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2020,8,31]]},"abstract":"<jats:p>Transferring the motion style from one animation clip to another, while preserving the motion content of the latter, has been a long-standing problem in character animation. Most existing data-driven approaches are supervised and rely on paired data, where motions with the same content are performed in different styles. In addition, these approaches are limited to transfer of styles that were seen during training.<\/jats:p>\n          <jats:p>In this paper, we present a novel data-driven framework for motion style transfer, which learns from an unpaired collection of motions with style labels, and enables transferring motion styles not observed during training. Furthermore, our framework is able to extract motion styles directly from videos, bypassing 3D reconstruction, and apply them to the 3D input motion.<\/jats:p>\n          <jats:p>Our style transfer network encodes motions into two latent codes, for content and for style, each of which plays a different role in the decoding (synthesis) process. While the content code is decoded into the output motion by several temporal convolutional layers, the style code modifies deep features via temporally invariant adaptive instance normalization (AdaIN).<\/jats:p>\n          <jats:p>Moreover, while the content code is encoded from 3D joint rotations, we learn a common embedding for style from either 3D or 2D joint positions, enabling style extraction from videos.<\/jats:p>\n          <jats:p>Our results are comparable to the state-of-the-art, despite not requiring paired training data, and outperform other methods when transferring previously unseen styles. To our knowledge, we are the first to demonstrate style transfer directly from videos to 3D animations - an ability which enables one to extend the set of style examples far beyond motions captured by MoCap systems.<\/jats:p>","DOI":"10.1145\/3386569.3392469","type":"journal-article","created":{"date-parts":[[2020,8,12]],"date-time":"2020-08-12T11:44:27Z","timestamp":1597232667000},"update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":160,"title":["Unpaired motion style transfer from video to animation"],"prefix":"10.1145","volume":"39","author":[{"given":"Kfir","family":"Aberman","sequence":"first","affiliation":[{"name":"Bejing Film Academy &amp; Tel-Aviv University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yijia","family":"Weng","sequence":"additional","affiliation":[{"name":"Peking University &amp; AICFVE"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dani","family":"Lischinski","sequence":"additional","affiliation":[{"name":"The Hebrew University of Jerusalem &amp; AICFVE"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Cohen-Or","sequence":"additional","affiliation":[{"name":"Tel-Aviv University &amp; AICFVE"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Baoquan","family":"Chen","sequence":"additional","affiliation":[{"name":"Peking University &amp; AICFVE"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2020,8,12]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Computer Graphics Forum","author":"Aberman Kfir","unstructured":"Kfir Aberman, Mingyi Shi, Jing Liao, Dani Lischinski, Baoquan Chen, and Daniel Cohen-Or. 2019a. Deep Video-Based Performance Cloning. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 219--233."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3322999"},{"key":"e_1_2_2_3_1","volume-title":"Graphics Interface","volume":"96","author":"Amaya Kenji","year":"1996","unstructured":"Kenji Amaya, Armin Bruderlin, and Tom Calvert. 1996. Emotion from motion. In Graphics Interface, Vol. 96. Toronto, Canada, 222--229."},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275038"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3099564.3099566"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/344779.344865"},{"key":"e_1_2_2_7_1","volume-title":"OpenPose: realtime multi-person 2D pose estimation using Part Affinity Fields. arXiv preprint arXiv:1812.08008","author":"Cao Zhe","year":"2018","unstructured":"Zhe Cao, Gines Hidalgo, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2018. OpenPose: realtime multi-person 2D pose estimation using Part Affinity Fields. arXiv preprint arXiv:1812.08008 (2018)."},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00603"},{"key":"e_1_2_2_9_1","unstructured":"CMU. 2019. CMU Graphics Lab Motion Capture Database. http:\/\/mocap.cs.cmu.edu\/"},{"key":"e_1_2_2_10_1","volume-title":"Proc. Eurographics. The Eurographics Association.","author":"Du Han","year":"2019","unstructured":"Han Du, Erik Herrmann, Janis Sprenger, Noshaba Cheema, Klaus Fischer, Philipp Slusallek, et al. 2019. Stylistic Locomotion Modeling with Conditional Variational Autoencoder. In Proc. Eurographics. The Eurographics Association."},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.265"},{"key":"e_1_2_2_12_1","volume-title":"ACM transactions on graphics (TOG)","author":"Grochow Keith","unstructured":"Keith Grochow, Steven L Martin, Aaron Hertzmann, and Zoran Popovi\u0107. 2004. Style-based inverse kinematics. In ACM transactions on graphics (TOG), Vol. 23. ACM, 522--531."},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2017.3271464"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073663"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925975"},{"key":"e_1_2_2_16_1","doi-asserted-by":"crossref","unstructured":"Daniel Holden Jun Saito Taku Komura and Thomas Joyce. 2015. Learning motion manifolds with convolutional autoencoders. In SIGGRAPH Asia 2015 Technical Briefs. ACM 18.","DOI":"10.1145\/2820903.2820918"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1186822.1073315"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.167"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01219-9_11"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1477926.1477927"},{"key":"e_1_2_2_21_1","volume-title":"Proc","author":"Johnson Justin","unstructured":"Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016. Perceptual losses for real-time style transfer and super-resolution. In Proc. ECCV. Springer, 694--711."},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00576"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00453"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1186822.1073314"},{"key":"e_1_2_2_25_1","volume-title":"Neural Rendering and Reenactment of Human Actor Videos. arXiv preprint arXiv:1809.03658","author":"Liu Lingjie","year":"2018","unstructured":"Lingjie Liu, Weipeng Xu, Michael Zollhoefer, Hyeongwoo Kim, Florian Bernard, Marc Habermann, Wenping Wang, and Christian Theobalt. 2018. Neural Rendering and Reenactment of Human Actor Videos. arXiv preprint arXiv:1809.03658 (2018)."},{"key":"e_1_2_2_26_1","volume-title":"Few-shot unsupervised image-to-image translation. arXiv preprint arXiv:1905.01723","author":"Liu Ming-Yu","year":"2019","unstructured":"Ming-Yu Liu, Xun Huang, Arun Mallya, Tero Karras, Timo Aila, Jaakko Lehtinen, and Jan Kautz. 2019. Few-shot unsupervised image-to-image translation. arXiv preprint arXiv:1905.01723 (2019)."},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/1921427.1921431"},{"key":"e_1_2_2_28_1","volume-title":"Computer Graphics Forum","author":"Mason Ian","unstructured":"Ian Mason, Sebastian Starke, He Zhang, Hakan Bilen, and Taku Komura. 2018. Few-shot Learning of Homogeneous Human Locomotion Styles. In Computer Graphics Forum, Vol. 37. Wiley Online Library, 143--153."},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073596"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00244"},{"key":"e_1_2_2_31_1","volume-title":"Modeling Human Motion with Quaternion-based Neural Networks. arXiv preprint arXiv:1901.07677","author":"Pavllo Dario","year":"2019","unstructured":"Dario Pavllo, Christoph Feichtenhofer, Michael Auli, and David Grangier. 2019a. Modeling Human Motion with Quaternion-based Neural Networks. arXiv preprint arXiv:1901.07677 (2019)."},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00794"},{"key":"e_1_2_2_33_1","volume-title":"Proc. Graphics Interface","author":"Shapiro Ari","year":"2006","unstructured":"Ari Shapiro, Yong Cao, and Petros Faloutsos. 2006. Style components. In Proc. Graphics Interface 2006. Canadian Information Processing Society, 33--39."},{"key":"e_1_2_2_34_1","volume-title":"Efficient Neural Networks for Real-time Motion Style Transfer. PACMCGIT 2, 2","author":"Smith Harrison Jesse","year":"2019","unstructured":"Harrison Jesse Smith, Chen Cao, Michael Neff, and Yingying Wang. 2019. Efficient Neural Networks for Real-time Motion Style Transfer. PACMCGIT 2, 2 (2019), 13:1--13:17."},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/1553374.1553505"},{"key":"e_1_2_2_36_1","volume-title":"Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022","author":"Ulyanov Dmitry","year":"2016","unstructured":"Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. 2016. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022 (2016)."},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/218380.218419"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00901"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/1273496.1273619"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00917"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766999"},{"key":"e_1_2_2_42_1","first-page":"137","article-title":"Spectral style transfer for human motion between independent actions","volume":"35","author":"Ersin Yumer M","year":"2016","unstructured":"M Ersin Yumer and Niloy J Mitra. 2016. Spectral style transfer for human motion between independent actions. ACM Transactions on Graphics (TOG) 35, 4 (2016), 137.","journal-title":"ACM Transactions on Graphics (TOG)"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00589"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3386569.3392469","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3386569.3392469","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,25]],"date-time":"2025-06-25T05:39:34Z","timestamp":1750829974000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3386569.3392469"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,12]]},"references-count":43,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2020,8,31]]}},"alternative-id":["10.1145\/3386569.3392469"],"URL":"https:\/\/doi.org\/10.1145\/3386569.3392469","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,8,12]]},"assertion":[{"value":"2020-08-12","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}