{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T08:55:03Z","timestamp":1790844903487,"version":"4.1.0"},"reference-count":58,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2024,3,8]],"date-time":"2024-03-08T00:00:00Z","timestamp":1709856000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"GPU cluster built by MCC Lab of Information Science and Technology Institution and the Supercomputing Center of the USTC"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2024,6,30]]},"abstract":"<jats:p>Reconstructing interacting hands from monocular RGB data is a challenging task, as it involves many interfering factors, e.g., self- and mutual occlusion and similar textures. Previous works only leverage information from a single RGB image without modeling their physically plausible relation, which leads to inferior reconstruction results. In this work, we are dedicated to explicitly exploiting spatial-temporal information to achieve better interacting hand reconstruction. On the one hand, we leverage temporal context to complement insufficient information provided by the single frame and design a novel temporal framework with a temporal constraint for interacting hand motion smoothness. On the other hand, we further propose an interpenetration detection module to produce kinetically plausible interacting hands without physical collisions. Extensive experiments are performed to validate the effectiveness of our proposed framework, which achieves new state-of-the-art performance on public benchmarks.<\/jats:p>","DOI":"10.1145\/3639707","type":"journal-article","created":{"date-parts":[[2024,1,24]],"date-time":"2024-01-24T12:17:11Z","timestamp":1706098631000},"page":"1-18","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Exploiting Spatial-Temporal Context for Interacting Hand Reconstruction on Monocular RGB Video"],"prefix":"10.1145","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7098-1690","authenticated-orcid":false,"given":"Weichao","family":"Zhao","sequence":"first","affiliation":[{"name":"University of Science and Technology of China, Hefei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0327-1562","authenticated-orcid":false,"given":"Hezhen","family":"Hu","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China, Hefei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1690-9836","authenticated-orcid":false,"given":"Wengang","family":"Zhou","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China, Hefei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7163-6263","authenticated-orcid":false,"given":"Li","family":"Li","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China, Hefei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2188-3028","authenticated-orcid":false,"given":"Houqiang","family":"Li","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China, Hefei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,3,8]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33783-3_46"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01110"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01231-1_41"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01031"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_3_1_7_2","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","volume":"34","author":"Diffenderfer James","year":"2021","unstructured":"James Diffenderfer, Brian Bartoldson, Shreya Chaganti, Jize Zhang, and Bhavya Kailkhura. 2021. A winning hand: Compressing deep networks can improve out-of-distribution robustness. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 34."},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2021.3058688"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/3DV53792.2021.00011"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00152"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2017.8297136"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392452"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00065"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01208"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_1_16_2","article-title":"SignBERT+: Hand-model-aware self-supervised pre-training for sign language understanding","author":"Hu Hezhen","year":"2023","unstructured":"Hezhen Hu, Weichao Zhao, Wengang Zhou, and Houqiang Li. 2023. SignBERT+: Hand-model-aware self-supervised pre-training for sign language understanding. IEEE Trans. Pattern Anal. Mach. Intell. 45, 9 (2023), 11221\u201311239.","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i2.16247"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3152129"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i07.6761"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01252-6_8"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3519305"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00576"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01100"},{"key":"e_1_3_1_24_2","article-title":"Adam: A method for stochastic optimization","author":"Kingma Diederik P.","year":"2014","unstructured":"Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv:1412.6980. Retrieved from https:\/\/arxiv.org\/abs\/1412.6980","journal-title":"arXiv:1412.6980"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.00278"},{"key":"e_1_3_1_26_2","doi-asserted-by":"crossref","unstructured":"Moran Li Yuan Gao and Nong Sang. 2021. Exploiting learnable joint groups for hand pose estimation. Proceedings of the AAAI Conference on Artificial Intelligence 35 3 (2021) 1921\u20131929.","DOI":"10.1609\/aaai.v35i3.16287"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2014.2363750"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACV48630.2021.00242"},{"key":"e_1_3_1_29_2","article-title":"HAN: An efficient hierarchical self-attention network for skeleton-based gesture recognition","author":"Liu Jianbo","year":"2021","unstructured":"Jianbo Liu, Ying Wang, Shiming Xiang, and Chunhong Pan. 2021. HAN: An efficient hierarchical self-attention network for skeleton-based gesture recognition. arXiv:2106.13391. Retrieved from https:\/\/arxiv.org\/abs\/2106.13391","journal-title":"arXiv:2106.13391"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-20068-7_22"},{"key":"e_1_3_1_31_2","first-page":"5079","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Moon Gyeongsik","year":"2018","unstructured":"Gyeongsik Moon, Ju Yong Chang, and Kyoung Mu Lee. 2018. V2v-posenet: Voxel-to-voxel prediction network for accurate 3D hand and human pose estimation from a single depth map. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5079\u20135088."},{"key":"e_1_3_1_32_2","first-page":"548","volume-title":"Proceedings of the European Conference on Computer Vision","author":"Moon Gyeongsik","year":"2020","unstructured":"Gyeongsik Moon, Shoou-I. Yu, He Wen, Takaaki Shiratori, and Kyoung Mu Lee. 2020. InterHand2. 6M: A dataset and baseline for 3D interacting hand pose estimation from a single RGB image. In Proceedings of the European Conference on Computer Vision. 548\u2013564."},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3322958"},{"key":"e_1_3_1_34_2","first-page":"7841","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","volume":"33","author":"Narasimhaswamy Supreeth","year":"2020","unstructured":"Supreeth Narasimhaswamy, Trung Nguyen, and Minh Hoai Nguyen. 2020. Detecting hands and recognizing physical contact in the wild. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 33. 7841\u20137851."},{"key":"e_1_3_1_35_2","first-page":"8026","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","volume":"32","author":"Paszke Adam","year":"2019","unstructured":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et\u00a0al. 2019. Pytorch: An imperative style, high-performance deep learning library. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 32. 8026\u20138037."},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.145"},{"key":"e_1_3_1_37_2","first-page":"458","volume-title":"Proceedings of IEEE International Conference on Robotics and Automation","author":"Romero Javier","year":"2010","unstructured":"Javier Romero, Hedvig Kjellstr\u00f6m, and Danica Kragic. 2010. Hands in action: Real-time 3D reconstruction of hands in interaction with objects. In Proceedings of IEEE International Conference on Robotics and Automation. 458\u2013463."},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3130800.3130883"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2017.2718622"},{"key":"e_1_3_1_40_2","first-page":"5898","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","volume":"34","author":"Shan Dandan","year":"2021","unstructured":"Dandan Shan, Richard Higgins, and David Fouhey. 2021. COHESIV: Contrastive object and hand embedding segmentation in video. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 34. 5898\u20135909."},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702179"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417768"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12700"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.380"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/2629500"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.90"},{"key":"e_1_3_1_47_2","first-page":"5998","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the Advances in Neural Information Processing Systems. 5998\u20136008."},{"key":"e_1_3_1_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01111"},{"issue":"6","key":"e_1_3_1_49_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3414685.3417852","article-title":"RGB2Hands: Real-time tracking of 3D hand interactions from monocular RGB video","volume":"39","author":"Wang Jiayi","year":"2020","unstructured":"Jiayi Wang, Franziska Mueller, Florian Bernard, Suzanne Sorli, Oleksandr Sotnychenko, Neng Qian, Miguel A. Otaduy, Dan Casas, and Christian Theobalt. 2020. RGB2Hands: Real-time tracking of 3D hand interactions from monocular RGB video. ACM Trans. Graph. 39, 6 (2020), 1\u201316.","journal-title":"ACM Trans. Graph."},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2462000"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2018.2879980"},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510004"},{"key":"e_1_3_1_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2020.2991987"},{"key":"e_1_3_1_54_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58610-2_8"},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01091"},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01116"},{"key":"e_1_3_1_57_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v37i3.25470"},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00539"},{"key":"e_1_3_1_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.525"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3639707","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3639707","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:03:49Z","timestamp":1750291429000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3639707"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,8]]},"references-count":58,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024,6,30]]}},"alternative-id":["10.1145\/3639707"],"URL":"https:\/\/doi.org\/10.1145\/3639707","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,8]]},"assertion":[{"value":"2023-03-28","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-12-15","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-03-08","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}