{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T10:49:58Z","timestamp":1782470998857,"version":"3.54.5"},"reference-count":24,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T00:00:00Z","timestamp":1782432000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Robot. AI"],"abstract":"<jats:p>Dexterous in-hand manipulation requires robotic hands to estimate object-state reliably under frequent occlusions, contact-rich interactions, and fast dynamics. Tactile sensing provides high-frequency, contact-specific feedback, although extracting useful representations from raw tactile signals and integrating them with vision and proprioception remains challenging. In this article, we present TouchWGNN, a multimodal dexterous manipulation framework that explicitly models tactile signals as a spatio-temporal graph. We first develop a low-cost distributed tactile sensor array for a five-fingered robotic hand, enabling real-time acquisition of normal forces from 113 sensing points distributed across key contact regions. We then construct a tactile graph in which taxels (or active contact points) are nodes with force features and 3D coordinates and edges encode spatial proximity and local force variation. A graph-based spatial encoder captures instantaneous contact geometry, and a temporal module models its evolution over time to refine object-state estimates. Finally, we fuse the tactile estimate with vision (point-cloud-based pose) and proprioception (joint states) for policy learning with reinforcement learning. Experiments on two in-hand manipulation tasks, cube reorientation and Baoding ball swapping, demonstrate that integrating raw tactile feedback with vision and proprioception improves manipulation performance compared with unimodal baselines.<\/jats:p>","DOI":"10.3389\/frobt.2026.1791424","type":"journal-article","created":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T10:08:56Z","timestamp":1782468536000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["TouchWGNN: spatio-temporal tactile perception for multimodal dexterous manipulation"],"prefix":"10.3389","volume":"13","author":[{"given":"Yu","family":"Ning","sequence":"first","affiliation":[{"name":"Wireless Multimedia Technology Lab, School of Computer Science and Technology, Dalian University of Technology","place":["Dalian, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"FuQiang","family":"Zhao","sequence":"additional","affiliation":[{"name":"Wireless Multimedia Technology Lab, School of Computer Science and Technology, Dalian University of Technology","place":["Dalian, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qian","family":"Liu","sequence":"additional","affiliation":[{"name":"Wireless Multimedia Technology Lab, School of Computer Science and Technology, Dalian University of Technology","place":["Dalian, China"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,6,26]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1177\/0278364919887447","article-title":"Learning dexterous in-hand manipulation","volume":"39","author":"Andrychowicz","year":"2020","journal-title":"Int. J. Robot. Res."},{"key":"B2","article-title":"Surprisingly robust in-hand manipulation: an empirical study","author":"Bhatt","year":"2022","journal-title":"CoRR abs\/2201.11503"},{"key":"B3","first-page":"14491","article-title":"How attentive are graph attention networks?","author":"Brody","year":"2021","journal-title":"CoRR abs\/2105"},{"key":"B4","first-page":"110","article-title":"Handrix: animating the human hand","author":"ElKoura","year":"2003"},{"key":"B5","first-page":"1","article-title":"Graph neural networks for interpretable tactile sensing","author":"Fan","year":"2022"},{"key":"B6","article-title":"Fast graph representation learning with pytorch geometric","author":"Fey","year":"2019","journal-title":"CoRR abs\/1903.02428"},{"key":"B7","first-page":"1","article-title":"Tactilegcn: a graph convolutional network for predicting grasp stability with tactile sensors","author":"Garcia-Garcia","year":"2019"},{"key":"B8","doi-asserted-by":"publisher","first-page":"9876","DOI":"10.1109\/IROS45743.2020.9341421","article-title":"TactileSGNet: a spiking graph neural network for event-based tactile object recognition,","author":"Gu","year":"2020","journal-title":"2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)"},{"key":"B9","first-page":"3142","article-title":"Dexterity from touch: self-supervised pre-training of tactile representations with robotic play,","volume":"229","author":"Guzey","year":"2023","journal-title":"Proceedings of The 7th Conference on Robot Learning, Proceedings of Machine Learning Research"},{"key":"B10","first-page":"9164","article-title":"Dexpilot: vision-based teleoperation of dexterous robotic hand-arm system","author":"Handa","year":"2020"},{"key":"B11","first-page":"6809","article-title":"Improving grasp stability with rotation measurement from tactile sensing","author":"Kolamuri","year":"2021"},{"key":"B12","doi-asserted-by":"publisher","first-page":"3894","DOI":"10.1109\/LRA.2024.3367271","article-title":"Tactile object property recognition using geometrical graph edge features and multi-thread graph convolutional network","volume":"9","author":"Kulkarni","year":"2024","journal-title":"IEEE Robot. Automation Lett."},{"key":"B13","doi-asserted-by":"publisher","first-page":"538773","DOI":"10.3389\/frobt.2021.538773","article-title":"Using tactile sensing to improve the sample efficiency and performance of deep deterministic policy gradients for simulated in-hand manipulation tasks","volume":"8","author":"Melnik","year":"2021","journal-title":"Front. Robot. AI"},{"key":"B14","doi-asserted-by":"publisher","first-page":"3994","DOI":"10.1109\/tro.2023.3280597","article-title":"Deep learning approaches to grasp synthesis: a review","volume":"39","author":"Newbury","year":"2023","journal-title":"IEEE Trans. Robot."},{"key":"B15","article-title":"Accelerating 3d deep learning with pytorch3d","author":"Ravi","year":"2020","journal-title":"arXiv:2007.08501"},{"key":"B16","first-page":"9323","article-title":"E(n) equivariant graph neural networks","volume":"139","author":"Satorras","year":"2021","journal-title":"Proceedings of the 38th International Conference on Machine Learning, Proceedings of Machine Learning Research"},{"key":"B17","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1109\/TNN.2008.2005605","article-title":"The graph neural network model","volume":"20","author":"Scarselli","year":"2009","journal-title":"IEEE Trans. Neural Netw."},{"key":"B18","first-page":"17868","article-title":"Foundationpose: unified 6d pose estimation and tracking of novel objects","author":"Wen","year":"2024"},{"key":"B19","article-title":"Semantically-regularized logic graph embeddings","author":"Xie","year":"2019","journal-title":"Corr. abs\/1909.01161"},{"key":"B20","doi-asserted-by":"publisher","first-page":"3605","DOI":"10.1109\/lra.2023.3264759","article-title":"Tacgnn: learning tactile-based in-hand manipulation with a blind robot using hierarchical graph neural network","volume":"8","author":"Yang","year":"2023","journal-title":"IEEE Robot. Automation Lett."},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2023.XIX.036","article-title":"Rotating without seeing: towards in-hand dexterity through touch,","author":"Yin","year":"2023","journal-title":"Proceedings of Robotics: Science and Systems"},{"key":"B22","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1016\/j.sna.2011.02.038","article-title":"Tactile sensing for dexterous in-hand manipulation in robotics\u2014a review","volume":"167","author":"Yousef","year":"2011","journal-title":"Sensors Actuators A Phys."},{"key":"B23","doi-asserted-by":"publisher","first-page":"21410","DOI":"10.1109\/jsen.2022.3210210","article-title":"Hardware technology of vision-based tactile sensor: a review","volume":"22","author":"Zhang","year":"2022","journal-title":"IEEE Sensors J."},{"key":"B24","article-title":"On the continuity of rotation representations in neural networks","author":"Zhou","year":"2018","journal-title":"Corr. abs\/1812.07035"}],"container-title":["Frontiers in Robotics and AI"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2026.1791424\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T10:08:58Z","timestamp":1782468538000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frobt.2026.1791424\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,26]]},"references-count":24,"alternative-id":["10.3389\/frobt.2026.1791424"],"URL":"https:\/\/doi.org\/10.3389\/frobt.2026.1791424","relation":{},"ISSN":["2296-9144"],"issn-type":[{"value":"2296-9144","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,26]]},"article-number":"1791424"}}