{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,28]],"date-time":"2026-07-28T22:15:19Z","timestamp":1785276919589,"version":"3.55.0"},"reference-count":53,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2023,10,31]],"date-time":"2023-10-31T00:00:00Z","timestamp":1698710400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2024,2,29]]},"abstract":"<jats:p>Accurate and reliable human motion reconstruction is crucial for creating natural interactions of full-body avatars in Virtual Reality (VR) and entertainment applications. As the Metaverse and social applications gain popularity, users are seeking cost-effective solutions to create full-body animations that are comparable in quality to those produced by commercial motion capture systems. In order to provide affordable solutions though, it is important to minimize the number of sensors attached to the subject\u2019s body. Unfortunately, reconstructing the full-body pose from sparse data is a heavily under-determined problem. Some studies that use IMU sensors face challenges in reconstructing the pose due to positional drift and ambiguity of the poses. In recent years, some mainstream VR systems have released 6-degree-of-freedom (6-DoF) tracking devices providing positional and rotational information. Nevertheless, most solutions for reconstructing full-body poses rely on traditional inverse kinematics (IK) solutions, which often produce non-continuous and unnatural poses. In this article, we introduce SparsePoser, a novel deep learning-based solution for reconstructing a full-body pose from a reduced set of six tracking devices. Our system incorporates a convolutional-based autoencoder that synthesizes high-quality continuous human poses by learning the human motion manifold from motion capture data. Then, we employ a learned IK component, made of multiple lightweight feed-forward neural networks, to adjust the hands and feet toward the corresponding trackers. We extensively evaluate our method on publicly available motion capture datasets and with real-time live demos. We show that our method outperforms state-of-the-art techniques using IMU sensors or 6-DoF tracking devices, and can be used for users with different body dimensions and proportions.<\/jats:p>","DOI":"10.1145\/3625264","type":"journal-article","created":{"date-parts":[[2023,10,9]],"date-time":"2023-10-09T12:17:04Z","timestamp":1696853824000},"page":"1-14","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":47,"title":["SparsePoser: Real-time Full-body Motion Reconstruction from Sparse Data"],"prefix":"10.1145","volume":"43","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6576-4528","authenticated-orcid":false,"given":"Jose Luis","family":"Ponton","sequence":"first","affiliation":[{"name":"Universitat Polit\u00e8cnica de Catalunya, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6192-6673","authenticated-orcid":false,"given":"Haoran","family":"Yun","sequence":"additional","affiliation":[{"name":"Universitat Polit\u00e8cnica de Catalunya, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7754-0791","authenticated-orcid":false,"given":"Andreas","family":"Aristidou","sequence":"additional","affiliation":[{"name":"University of Cyprus, Cyprus and CYENS Centre of Excellence, Cyprus"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8480-4713","authenticated-orcid":false,"given":"Carlos","family":"Andujar","sequence":"additional","affiliation":[{"name":"Universitat Polit\u00e8cnica de Catalunya, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1437-245X","authenticated-orcid":false,"given":"Nuria","family":"Pelechano","sequence":"additional","affiliation":[{"name":"Universitat Polit\u00e8cnica de Catalunya, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,10,31]]},"reference":[{"key":"e_1_3_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392462"},{"key":"e_1_3_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3463499"},{"key":"e_1_3_2_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01290"},{"key":"e_1_3_2_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2022.3181374"},{"key":"e_1_3_2_6_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14632"},{"key":"e_1_3_2_7_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13310"},{"key":"e_1_3_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3344383"},{"key":"e_1_3_2_9_1","unstructured":"Raphael Bensadoun Shir Gur Nitsan Blau and Lior Wolf. 2022. Neural inverse kinematic. In Proceedings of the 39th International Conference on Machine Learning (Proceedings of Machine Learning Research) PMLR 1787\u20131797."},{"key":"e_1_3_2_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2011.6094666"},{"key":"e_1_3_2_11_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10055-018-0374-z"},{"key":"e_1_3_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3026276"},{"key":"e_1_3_2_13_1","volume-title":"Proceedings of the Game Developers Conference.","author":"Clavet Simon","year":"2016","unstructured":"Simon Clavet. 2016. Motion matching and the road to next-gen animation. In Proceedings of the Game Developers Conference."},{"key":"e_1_3_2_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/M2VIP.2017.8211457"},{"key":"e_1_3_2_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3025175"},{"key":"e_1_3_2_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01148"},{"key":"e_1_3_2_17_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.protcy.2013.12.451"},{"key":"e_1_3_2_18_1","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.05-07-01688.1985"},{"key":"e_1_3_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.2973077"},{"key":"e_1_3_2_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01093"},{"key":"e_1_3_2_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10055-021-00530-5"},{"key":"e_1_3_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015706.1015755"},{"key":"e_1_3_2_23_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13089"},{"key":"e_1_3_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275108"},{"key":"e_1_3_2_25_1","unstructured":"Yan-Bin Jia. 2013. Dual Quaternions. (2013). Retrieved fromhttps:\/\/faculty.sites.iastate.edu\/jia\/files\/inline-files\/dual-quaternion.pdf. Access 9th December 2022."},{"key":"e_1_3_2_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-20065-6_26"},{"key":"e_1_3_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3550469.3555428"},{"key":"e_1_3_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1230100.1230107"},{"key":"e_1_3_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2816795.2818013"},{"key":"e_1_3_2_30_1","unstructured":"Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations . Retrieved from https:\/\/openreview.net\/forum?id=Bkg6RiCqY7"},{"key":"e_1_3_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00554"},{"key":"e_1_3_2_32_1","doi-asserted-by":"publisher","DOI":"10.3389\/frvir.2022.937191"},{"key":"e_1_3_2_33_1","volume-title":"Proceedings of theAdvances in Neural Information Processing Systems","author":"Paszke Adam","year":"2019","unstructured":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An imperative style, high-performance deep learning library. In Proceedings of theAdvances in Neural Information Processing Systems. Curran Associates, Inc.Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2019\/hash\/bdbca288fee7f92f2bfa9f7012727740-Abstract.html"},{"key":"e_1_3_2_34_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-019-01245-6"},{"key":"e_1_3_2_35_1","volume-title":"Proceedings of the British Machine Vision Conference.","author":"Pavllo Dario","year":"2018","unstructured":"Dario Pavllo, David Grangier, and Michael Auli. 2018. QuaterNet: A quaternion-based recurrent model for human motion. In Proceedings of the British Machine Vision Conference."},{"key":"e_1_3_2_36_1","doi-asserted-by":"publisher","DOI":"10.2312\/egs20221037"},{"key":"e_1_3_2_37_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14628"},{"key":"e_1_3_2_38_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2019.103386"},{"key":"e_1_3_2_39_1","unstructured":"RootMotion. 2017. Final IK. (2017). Retrieved from http:\/\/root-motion.com\/. Access 9th December 2022."},{"key":"e_1_3_2_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/VR.2019.8798108"},{"key":"e_1_3_2_41_1","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","volume":"30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the Advances in Neural Information Processing Systems. I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30, Curran Associates, Inc."},{"key":"e_1_3_2_42_1","doi-asserted-by":"publisher","DOI":"10.1002\/cav.2013"},{"key":"e_1_3_2_43_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13131"},{"key":"e_1_3_2_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3550469.3555411"},{"key":"e_1_3_2_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2009.111"},{"key":"e_1_3_2_46_1","unstructured":"Xsens. 2000. 3D motion tracking. (2000). https:\/\/www.xsens.com\/. Access 9th December 2022."},{"key":"e_1_3_2_47_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.142631"},{"key":"e_1_3_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3592099"},{"key":"e_1_3_2_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01282"},{"key":"e_1_3_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459786"},{"key":"e_1_3_2_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/VR55154.2023.00044"},{"key":"e_1_3_2_52_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10055-022-00635-5"},{"key":"e_1_3_2_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00589"},{"key":"e_1_3_2_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00539"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3625264","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3625264","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:50:04Z","timestamp":1750287004000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3625264"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,31]]},"references-count":53,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,2,29]]}},"alternative-id":["10.1145\/3625264"],"URL":"https:\/\/doi.org\/10.1145\/3625264","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,31]]},"assertion":[{"value":"2023-04-27","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-09-14","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-10-31","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}