{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T08:05:56Z","timestamp":1790841956763,"version":"4.1.0"},"reference-count":80,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2022,11,30]],"date-time":"2022-11-30T00:00:00Z","timestamp":1669766400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2022,12]]},"abstract":"<jats:p>We present a deep learning-based framework to synthesize motion in-betweening in a two-stage manner. Given some context frames and a target frame, the system can generate plausible transitions with variable lengths in a non-autoregressive fashion. The framework consists of two Transformer Encoder-based networks operating in two stages: in the first stage a Context Transformer is designed to generate rough transitions based on the context and in the second stage a Detail Transformer is employed to refine motion details. Compared to existing Transformer-based methods which either use a complete Transformer Encoder-Decoder architecture or additional 1D convolutions to generate motion transitions, our framework achieves superior performance with less trainable parameters by only leveraging the Transformer Encoder and masked self-attention mechanism. To enhance the generalization of our transformer-based framework, we further introduce Keyframe Positional Encoding and Learned Relative Positional Encoding to make our method robust in synthesizing longer transitions exceeding the maximum transition length during training. Our framework is also artist-friendly by supporting full and partial pose constraints within the transition, giving artists fine control over the synthesized results. We benchmark our framework on the LAFAN1 dataset, and experiments show that our method outperforms the current state-of-the-art methods at a large margin (an average of 16% for normal-length sequences and 55% for excessive-length sequences). Our method trains faster than the RNN-based method and achieves a four-time speedup during inference. We implement our framework into a production-ready tool inside an animation authoring software and conduct a pilot study to validate the practical value of our method.<\/jats:p>","DOI":"10.1145\/3550454.3555454","type":"journal-article","created":{"date-parts":[[2022,11,30]],"date-time":"2022-11-30T21:19:07Z","timestamp":1669843147000},"page":"1-16","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":58,"title":["Motion In-Betweening via Two-Stage Transformers"],"prefix":"10.1145","volume":"41","author":[{"given":"Jia","family":"Qin","sequence":"first","affiliation":[{"name":"Zhejiang University and Netease Fuxi AI Lab"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Youyi","family":"Zheng","sequence":"additional","affiliation":[{"name":"Zhejiang University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kun","family":"Zhou","sequence":"additional","affiliation":[{"name":"Zhejiang University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,11,30]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/3DV53792.2021.00066"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/566654.566606"},{"key":"e_1_2_2_3_1","unstructured":"Autodesk. 2022. Maya. https:\/\/www.autodesk.com\/products\/maya\/overview"},{"key":"e_1_2_2_4_1","volume-title":"Jamie Ryan Kiros, and Geoffrey E Hinton","author":"Ba Jimmy Lei","year":"2016","unstructured":"Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. Layer normalization. arXiv preprint arXiv:1607.06450 (2016)."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW.2018.00191"},{"key":"e_1_2_2_6_1","volume-title":"Proceedings of the 2008 ACM SIGGRAPH\/Eurographics Symposium on Computer Animation. 117--126","author":"Beaudoin Philippe","unstructured":"Philippe Beaudoin, Stelian Coros, Michiel van de Panne, and Pierre Poulin. 2008. Motionmotif graphs. In Proceedings of the 2008 ACM SIGGRAPH\/Eurographics Symposium on Computer Animation. 117--126."},{"key":"e_1_2_2_7_1","unstructured":"Blender Foundation. 2022. Blender. http:\/\/www.blender.org"},{"key":"e_1_2_2_8_1","unstructured":"Michael B\u00fcttner and Simon Clavet. 2015. Motion Matching. https:\/\/www.youtube.com\/watch?v=z_wpgHFSWss&t=658s."},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58571-6_14"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01144"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073248"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276387"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1008"},{"key":"e_1_2_2_14_1","volume-title":"2019 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 1423--1432","author":"Adeli Ehsan","year":"2019","unstructured":"Hsu-kuang Chiu, Ehsan Adeli, Borui Wang, De-An Huang, and Juan Carlos Niebles. 2019. Action-agnostic human pose forecasting. In 2019 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 1423--1432."},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3322938"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1285"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01127"},{"key":"e_1_2_2_18_1","volume-title":"Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805","author":"Devlin Jacob","year":"2018","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)."},{"key":"e_1_2_2_19_1","volume-title":"Single-Shot Motion Completion with Transformer. arXiv preprint arXiv:2103.00776","author":"Duan Yinglin","year":"2021","unstructured":"Yinglin Duan, Tianyang Shi, Zhengxia Zou, Yenan Lin, Zhehui Qian, Bohan Zhang, and Yi Yuan. 2021. Single-Shot Motion Completion with Transformer. arXiv preprint arXiv:2103.00776 (2021)."},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383287"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/882262.882286"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.494"},{"key":"e_1_2_2_23_1","volume-title":"International Conference on Machine Learning. PMLR, 1243--1252","author":"Gehring Jonas","year":"2017","unstructured":"Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin. 2017. Convolutional sequence to sequence learning. In International Conference on Machine Learning. PMLR, 1243--1252."},{"key":"e_1_2_2_24_1","volume-title":"Lightweight Quaternion Transition Generation with Neural Networks. In 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, 579--580","author":"Geleijn Romi","year":"2021","unstructured":"Romi Geleijn, Adrian Radziszewski, Julia Beryl van Straaten, and Henrique Galvan Debarba. 2021. Lightweight Quaternion Transition Generation with Neural Networks. In 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, 579--580."},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01239"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015706.1015755"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01225-0_48"},{"key":"e_1_2_2_28_1","doi-asserted-by":"crossref","unstructured":"F\u00e9lix G Harvey and Christopher Pal. 2018. Recurrent transition networks for character locomotion. In SIGGRAPH Asia 2018 Technical Briefs. 1--4.","DOI":"10.1145\/3283254.3283277"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392480"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.123"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00723"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/218380.218414"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392440"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073663"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925975"},{"key":"e_1_2_2_37_1","doi-asserted-by":"crossref","unstructured":"Daniel Holden Jun Saito Taku Komura and Thomas Joyce. 2015. Learning motion manifolds with convolutional autoencoders. In SIGGRAPH Asia 2015 technical briefs. 1--4.","DOI":"10.1145\/2820903.2820918"},{"key":"e_1_2_2_38_1","volume-title":"Music transformer. arXiv preprint arXiv:1809.04281","author":"Anna Huang Cheng-Zhi","year":"2018","unstructured":"Cheng-Zhi Anna Huang, Ashish Vaswani, Jakob Uszkoreit, Noam Shazeer, Ian Simon, Curtis Hawthorne, Andrew M Dai, Matthew D Hoffman, Monica Dinculescu, and Douglas Eck. 2018. Music transformer. arXiv preprint arXiv:1809.04281 (2018)."},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.573"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/3DV50981.2020.00102"},{"key":"e_1_2_2_41_1","volume-title":"Conditional Motion In-betweening. arXiv preprint arXiv:2202.04307","author":"Kim Jihoon","year":"2022","unstructured":"Jihoon Kim, Taehyun Byun, Seungyoun Shin, Jungdam Won, and Sungjoon Choi. 2022. Conditional Motion In-betweening. arXiv preprint arXiv:2202.04307 (2022)."},{"key":"e_1_2_2_42_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980","author":"Kingma Diederik P","year":"2014","unstructured":"Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)."},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015706.1015760"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/566654.566605"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/566570.566607"},{"key":"e_1_2_2_46_1","doi-asserted-by":"crossref","unstructured":"Yongjoon Lee Kevin Wampler Gilbert Bernstein Jovan Popovi\u0107 and Zoran Popovi\u0107. 2010. Motion fields for interactive character locomotion. In ACM SIGGRAPH Asia 2010 papers. 1--8.","DOI":"10.1145\/1882261.1866160"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.171"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/3DV53792.2021.00086"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392422"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2020.3021409"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.497"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2366145.2366172"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/1640443.1640452"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073313"},{"key":"e_1_2_2_55_1","volume-title":"Proceedings of the 20th annual conference on Computer graphics and interactive techniques. 343--350","author":"Thomas Ngo J","year":"1993","unstructured":"J Thomas Ngo and Joe Marks. 1993. Spacetime constraints revisited. In Proceedings of the 20th annual conference on Computer graphics and interactive techniques. 343--350."},{"key":"e_1_2_2_56_1","volume-title":"Motion Inbetweening via Deep \u0394-Interpolator. arXiv preprint arXiv:2201.06701","author":"Oreshkin Boris N","year":"2022","unstructured":"Boris N Oreshkin, Antonios Valkanas, F\u00e9lix G Harvey, Louis-Simon M\u00e9nard, Florent Bocquelet, and Mark J Coates. 2022. Motion Inbetweening via Deep \u0394-Interpolator. arXiv preprint arXiv:2201.06701 (2022)."},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/545261.545279"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-019-01245-6"},{"key":"e_1_2_2_59_1","volume-title":"Quaternet: A quaternion-based recurrent model for human motion. arXiv preprint arXiv:1805.06485","author":"Pavllo Dario","year":"2018","unstructured":"Dario Pavllo, David Grangier, and Michael Auli. 2018. Quaternet: A quaternion-based recurrent model for human motion. arXiv preprint arXiv:1805.06485 (2018)."},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01080"},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/38.708559"},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/237170.237229"},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276510"},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-2074"},{"key":"e_1_2_2_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356505"},{"key":"e_1_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392450"},{"key":"e_1_2_2_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459881"},{"key":"e_1_2_2_68_1","volume-title":"Real-time Controllable Motion Transition for Characters. arXiv preprint arXiv:2205.02540","author":"Tang Xiangjun","year":"2022","unstructured":"Xiangjun Tang, He Wang, Bo Hu, Xu Gong, Ruifan Yi, Qilong Kou, and Xiaogang Jin. 2022. Real-time Controllable Motion Transition for Characters. arXiv preprint arXiv:2205.02540 (2022)."},{"key":"e_1_2_2_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/1966394.1966397"},{"key":"e_1_2_2_70_1","volume-title":"Modeling human motion using binary latent variables. Advances in neural information processing systems 19","author":"Taylor Graham W","year":"2006","unstructured":"Graham W Taylor, Geoffrey E Hinton, and Sam Roweis. 2006. Modeling human motion using binary latent variables. Advances in neural information processing systems 19 (2006)."},{"key":"e_1_2_2_71_1","unstructured":"Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008."},{"key":"e_1_2_2_72_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00901"},{"key":"e_1_2_2_73_1","volume-title":"Gaussian process dynamical models for human motion","author":"Wang Jack M","year":"2007","unstructured":"Jack M Wang, David J Fleet, and Aaron Hertzmann. 2007. Gaussian process dynamical models for human motion. IEEE transactions on pattern analysis and machine intelligence 30, 2 (2007), 283--298."},{"key":"e_1_2_2_74_1","volume-title":"Learning deep transformer models for machine translation. arXiv preprint arXiv:1906.01787","author":"Wang Qiang","year":"2019","unstructured":"Qiang Wang, Bei Li, Tong Xiao, Jingbo Zhu, Changliang Li, Derek F Wong, and Lidia S Chao. 2019. Learning deep transformer models for machine translation. arXiv preprint arXiv:1906.01787 (2019)."},{"key":"e_1_2_2_75_1","volume-title":"Unicon: Universal neural controller for physics-based character motion. arXiv preprint arXiv:2011.15119","author":"Wang Tingwu","year":"2020","unstructured":"Tingwu Wang, Yunrong Guo, Maria Shugrina, and Sanja Fidler. 2020. Unicon: Universal neural controller for physics-based character motion. arXiv preprint arXiv:2011.15119 (2020)."},{"key":"e_1_2_2_76_1","doi-asserted-by":"publisher","DOI":"10.1145\/54852.378507"},{"key":"e_1_2_2_77_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275090"},{"key":"e_1_2_2_78_1","volume-title":"Proceedings of the 11th Annual International Conference on Motion, Interaction, and Games. 1--11","author":"Zhang Xinyi","unstructured":"Xinyi Zhang and Michiel van de Panne. 2018. Data-driven autocompletion for keyframe animation. In Proceedings of the 11th Annual International Conference on Motion, Interaction, and Games. 1--11."},{"key":"e_1_2_2_79_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00589"},{"key":"e_1_2_2_80_1","unstructured":"Yi Zhou Jingwan Lu Connelly Barnes Jimei Yang Sitao Xiang et al. 2020. Generative tweening: Long-term inbetweening of 3d human motions. arXiv preprint arXiv:2005.08891 (2020)."}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3550454.3555454","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3550454.3555454","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:49:11Z","timestamp":1750182551000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3550454.3555454"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,30]]},"references-count":80,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["10.1145\/3550454.3555454"],"URL":"https:\/\/doi.org\/10.1145\/3550454.3555454","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,30]]},"assertion":[{"value":"2022-11-30","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}