{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,15]],"date-time":"2026-06-15T17:53:51Z","timestamp":1781546031004,"version":"3.54.5"},"reference-count":71,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2026,6,15]],"date-time":"2026-06-15T00:00:00Z","timestamp":1781481600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/legalcode"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62302168"],"award-info":[{"award-number":["62302168"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100012166","name":"National Key R&D Program of China","doi-asserted-by":"crossref","award":["2022YFB4500603"],"award-info":[{"award-number":["2022YFB4500603"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Guangdong Provincial Key Laboratory of Human Digital Twin","award":["2022B1212010004"],"award-info":[{"award-number":["2022B1212010004"]}]},{"name":"Guangzhou Basic Research Program","award":["SL2023A04J00930"],"award-info":[{"award-number":["SL2023A04J00930"]}]},{"name":"Shenzhen Holdfound Foundation Endowed Professorship","award":["N\/A"],"award-info":[{"award-number":["N\/A"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2026,6,15]]},"abstract":"<jats:p>Hand-object interactions are central to everyday activities, yet most intelligent assistants today remain blind to users' physical actions. Existing IMU-based recognition approaches focus on classifying predefined gestures, but they lack the semantic expressiveness required for contextual support in real-world scenarios such as office work and home routines. In this paper, we introduce a semantic tokenization pipeline that bridges continuous inertial signals and large language models (LLMs), enabling assistants to \u201cread\u201d hand movements as naturally as words. We first collected a multimodal dataset of dual-hand activities across office and home environments capturing long-horizon action chains that span multiple interrelated sub-tasks. Using self-supervised representation learning, we discretize IMU embeddings into action tokens that approximate a vocabulary of hand interactions. These tokens are then aligned with natural language through instruction-tuned LLMs, supporting tasks such as action captioning, intent inference, and contextual feedback. Evaluation shows that our tokenization improves semantic consistency with language distributions, and the LLM produces accurate, human-preferred descriptions of actions across diverse activities. We further demonstrate a proof-of-concept assistant prototype that generates contextual reminders. Our findings highlight the potential of transforming raw hand motions into a \u201clanguage of actions,\u201d paving the way for everyday intelligent assistants that are aware of users' physical interactions. The Project page, source code, and dataset are publicly available at https:\/\/scut-hai.github.io\/HMotionGPT\/.<\/jats:p>","DOI":"10.1145\/3810222","type":"journal-article","created":{"date-parts":[[2026,6,15]],"date-time":"2026-06-15T17:06:41Z","timestamp":1781543201000},"page":"1-33","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["HMotionGPT: Aligning Hand Motions and Natural Language for Activity Understanding with Smart Rings"],"prefix":"10.1145","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6811-0183","authenticated-orcid":false,"given":"Yang","family":"Gao","sequence":"first","affiliation":[{"name":"School of Future Technology, South China University of Technology, Guangzhou, China and School of Computer Science and Technology, East China Normal University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-5378-0505","authenticated-orcid":false,"given":"Dong","family":"She","sequence":"additional","affiliation":[{"name":"School of Future Technology, South China University of Technology, Guangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-4569-3448","authenticated-orcid":false,"given":"Wolin","family":"Liang","sequence":"additional","affiliation":[{"name":"South China University of Technology, Guangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-3825-9944","authenticated-orcid":false,"given":"Chiyue","family":"Wang","sequence":"additional","affiliation":[{"name":"South China University of Technology, Guangzhou, Guangdong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-9380-2922","authenticated-orcid":false,"given":"Yingjing","family":"Xiao","sequence":"additional","affiliation":[{"name":"East China Normal University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-1965-4499","authenticated-orcid":false,"given":"Xianrong","family":"Yao","sequence":"additional","affiliation":[{"name":"South China University of Technology, Guangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-9128-7578","authenticated-orcid":false,"given":"Cong","family":"Liu","sequence":"additional","affiliation":[{"name":"South China University of Technology, Guangzhou, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-5609-687X","authenticated-orcid":false,"given":"Zhichao","family":"Huang","sequence":"additional","affiliation":[{"name":"East China Normal University, Shanghai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3020-3736","authenticated-orcid":false,"given":"Zhanpeng","family":"Jin","sequence":"additional","affiliation":[{"name":"School of Future Technology, South China University of Technology, Guangzhou, Guangdong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,6,15]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation. arXiv preprint arXiv:1308.3432","author":"Bengio Yoshua","year":"2013","unstructured":"Yoshua Bengio, Nicholas L\u00e9onard, and Aaron Courville. 2013. Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation. arXiv preprint arXiv:1308.3432 (2013)."},{"key":"e_1_2_1_2_1","volume-title":"Towards Generalizable Human Activity Recognition: A Survey. arXiv preprint arXiv:2508.12213","author":"Cai Yize","year":"2025","unstructured":"Yize Cai, Baoshen Guo, Flora Salim, and Zhiqing Hong. 2025. Towards Generalizable Human Activity Recognition: A Survey. arXiv preprint arXiv:2508.12213 (2025)."},{"key":"e_1_2_1_3_1","doi-asserted-by":"crossref","unstructured":"Alessandro Carf\u00ec Mohamad Alameh Valerio Belcamino and Fulvio Mastrogiovanni. 2024. A modular architecture for IMU-based data gloves. arXiv:2401.13254 [cs.AR] https:\/\/arxiv.org\/abs\/2401.13254","DOI":"10.1007\/978-3-031-76424-0_10"},{"key":"e_1_2_1_4_1","doi-asserted-by":"crossref","first-page":"110731","DOI":"10.1016\/j.dib.2024.110731","article-title":"UMAHand: A dataset of inertial signals of typical hand activities","volume":"55","author":"Casilari Eduardo","year":"2024","unstructured":"Eduardo Casilari, Jennifer Barbosa-Galeano, and Francisco J Gonz\u00e1lez-Ca\u00f1ete. 2024. UMAHand: A dataset of inertial signals of typical hand activities. Data in Brief 55 (2024), 110731.","journal-title":"Data in Brief"},{"key":"e_1_2_1_5_1","volume-title":"Human Activity Recognition and Behaviour Analysis: For Cyber-Physical Systems in Smart Environments","author":"Chen Liming","unstructured":"Liming Chen and Chris D Nugent. 2019. Composite activity recognition. In Human Activity Recognition and Behaviour Analysis: For Cyber-Physical Systems in Smart Environments. Springer, 151\u2013181."},{"key":"e_1_2_1_6_1","volume-title":"Motionllm: Understanding human behaviors from human motions and videos. arXiv preprint arXiv:2405.20340","author":"Chen Ling-Hao","year":"2024","unstructured":"Ling-Hao Chen, Shunlin Lu, Ailing Zeng, Hao Zhang, Benyou Wang, Ruimao Zhang, and Lei Zhang. 2024. Motionllm: Understanding human behaviors from human motions and videos. arXiv preprint arXiv:2405.20340 (2024)."},{"key":"e_1_2_1_7_1","volume-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 8, 4, Article 192 (Nov.","author":"Chen Wenqiang","year":"2024","unstructured":"Wenqiang Chen, Jiaxuan Cheng, Leyao Wang, Wei Zhao, and Wojciech Matusik. 2024. Sensor2Text: Enabling Natural Language Interactions for Daily Activity Tracking Using Wearable Sensors. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 8, 4, Article 192 (Nov. 2024), 26 pages. https:\/\/doi.org\/10.1145\/3699747"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2022.3191173"},{"key":"e_1_2_1_9_1","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1109\/70.34763","article-title":"On grasp choice, grasp models, and the design of hands for manufacturing tasks","volume":"5","author":"Cutkosky Mark R","year":"1989","unstructured":"Mark R Cutkosky et al. 1989. On grasp choice, grasp models, and the design of hands for manufacturing tasks. IEEE Transactions on robotics and automation 5, 3 (1989), 269\u2013279.","journal-title":"IEEE Transactions on robotics and automation"},{"key":"e_1_2_1_10_1","volume-title":"CHI 2009 Workshop. Developing Shared Home Behavior Datasets to Advance HCI and Ubiquitous Computing Research.","author":"de la Torre Fernando","year":"2009","unstructured":"Fernando de la Torre, Jessica K. Hodgins, Javier Montano, and Sergio Valcarcel. 2009. Detailed Human Data Acquisition of Kitchen Activities: the CMU-Multimodal Activity Database (CMU-MMAC). In CHI 2009 Workshop. Developing Shared Home Behavior Datasets to Advance HCI and Ubiquitous Computing Research."},{"key":"e_1_2_1_11_1","volume-title":"Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track","volume":"3","author":"DelPreto Joseph","year":"2022","unstructured":"Joseph DelPreto, Chao Liu, Yiyue Luo, Michael Foshey, Yunzhu Li, Antonio Torralba, Wojciech Matusik, and Daniela Rus. 2022. Actionnet: A multimodal dataset for human activities using wearable sensors in a kitchen environment. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, Vol. 3."},{"key":"e_1_2_1_12_1","doi-asserted-by":"crossref","first-page":"275","DOI":"10.1109\/TCSVT.2021.3058688","article-title":"Understanding atomic hand-object interaction with human intention","volume":"32","author":"Fan Hehe","year":"2021","unstructured":"Hehe Fan, Tao Zhuo, Xin Yu, Yi Yang, and Mohan Kankanhalli. 2021. Understanding atomic hand-object interaction with human intention. IEEE Transactions on Circuits and Systems for Video Technology 32, 1 (2021), 275\u2013285.","journal-title":"IEEE Transactions on Circuits and Systems for Video Technology"},{"key":"e_1_2_1_13_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 3293\u20133303","author":"Goyal Mohit","year":"2022","unstructured":"Mohit Goyal, Sahil Modi, Rishabh Goyal, and Saurabh Gupta. 2022. Human hands as probes for interactive object understanding. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 3293\u20133303."},{"key":"e_1_2_1_14_1","volume-title":"First conference on language modeling.","author":"Gu Albert","year":"2024","unstructured":"Albert Gu and Tri Dao. 2024. Mamba: Linear-time sequence modeling with selective state spaces. In First conference on language modeling."},{"key":"e_1_2_1_15_1","volume-title":"Ankit Kumar Das, and Ghanapriya Singh","author":"Gupta Rinki","year":"2025","unstructured":"Rinki Gupta, Ankit Kumar Das, and Ghanapriya Singh. 2025. Attention-Based Deep Learning for Hand Gesture Recognition Using Multi-sensor Data. In Artificial Intelligence: Towards Sustainable Intelligence, Sanju Tiwari, Fernando Ortiz-Rodriguez, Miguel-Angel Sicilia, and Tek Raj Chhetri (Eds.). Springer Nature Switzerland, Cham, 115\u2013126."},{"key":"e_1_2_1_16_1","doi-asserted-by":"crossref","first-page":"100046","DOI":"10.1016\/j.jjimei.2021.100046","article-title":"Deep learning based human activity recognition (HAR) using wearable sensor data","volume":"1","author":"Gupta Saurabh","year":"2021","unstructured":"Saurabh Gupta. 2021. Deep learning based human activity recognition (HAR) using wearable sensor data. International Journal of Information Management Data Insights 1, 2 (2021), 100046.","journal-title":"International Journal of Information Management Data Insights"},{"key":"e_1_2_1_17_1","volume-title":"Gunar Schirner, Ta\u015fkin Padir, and Deniz Erdo\u011fmu\u015f.","author":"Han Mo","year":"2020","unstructured":"Mo Han, Sezen Ya\u011fmur G\u00fcnay, Gunar Schirner, Ta\u015fkin Padir, and Deniz Erdo\u011fmu\u015f. 2020. HANDS: a multimodal dataset for modeling toward human grasp intent inference in prosthetic hands. Intelligent service robotics 13, 1 (2020), 179\u2013185."},{"key":"e_1_2_1_18_1","volume-title":"Proceedings of the IEEE conference on computer vision and pattern recognition. 770\u2013778","author":"He Kaiming","year":"2016","unstructured":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770\u2013778."},{"key":"e_1_2_1_19_1","volume-title":"Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI '25)","author":"He Zhe","year":"2025","unstructured":"Zhe He, Zixuan Wang, Chun Yu, Chengwen Zhang, Xiyuan Shen, and Yuanchun Shi. 2025. WritingRing: Enabling Natural Handwriting Input with a Single IMU Ring. In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI '25). Association for Computing Machinery, New York, NY, USA, Article 731, 15 pages. https:\/\/doi.org\/10.1145\/3706598.3714066"},{"key":"e_1_2_1_20_1","volume-title":"A comparison of complementary and Kalman filtering","author":"Higgins Walter T","year":"2007","unstructured":"Walter T Higgins. 2007. A comparison of complementary and Kalman filtering. IEEE transactions on aerospace and electronic systems 3 (2007), 321\u2013325."},{"key":"e_1_2_1_21_1","volume-title":"Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2. 4511\u20134521","author":"Hong Zhiqing","year":"2025","unstructured":"Zhiqing Hong, Yiwei Song, Zelong Li, Anlan Yu, Shuxin Zhong, Yi Ding, Tian He, and Desheng Zhang. 2025. Llm4har: Generalizable on-device human activity recognition with pretrained llms. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2. 4511\u20134521."},{"key":"e_1_2_1_22_1","volume-title":"The representing brain: Neural correlates of motor intention and imagery. Behavioral and Brain sciences 17, 2","author":"Jeannerod Marc","year":"1994","unstructured":"Marc Jeannerod. 1994. The representing brain: Neural correlates of motor intention and imagery. Behavioral and Brain sciences 17, 2 (1994), 187\u2013202."},{"key":"e_1_2_1_23_1","first-page":"20067","article-title":"Motiongpt: Human motion as a foreign language","volume":"36","author":"Jiang Biao","year":"2023","unstructured":"Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, and Tao Chen. 2023. Motiongpt: Human motion as a foreign language. Advances in Neural Information Processing Systems 36 (2023), 20067\u201320079.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.3390\/s22155855"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/JBHI.2021.3100099"},{"key":"e_1_2_1_26_1","volume-title":"International conference on machine learning. PMLR, 792\u2013800","author":"Koppula Hema","year":"2013","unstructured":"Hema Koppula and Ashutosh Saxena. 2013. Learning spatio-temporal structure from rgb-d videos for human activity detection and anticipation. In International conference on machine learning. PMLR, 792\u2013800."},{"key":"e_1_2_1_27_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3678545","article-title":"Imugpt 2.0: Language-based cross modality transfer for sensor-based human activity recognition","volume":"8","author":"Leng Zikang","year":"2024","unstructured":"Zikang Leng, Amitrajit Bhattacharjee, Hrudhai Rajasekhar, Lizhe Zhang, Elizabeth Bruda, Hyeokhyen Kwon, and Thomas Pl\u00f6tz. 2024. Imugpt 2.0: Language-based cross modality transfer for sensor-based human activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8, 3 (2024), 1\u201332.","journal-title":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"},{"key":"e_1_2_1_28_1","volume-title":"Sensorllm: Aligning large language models with motion sensors for human activity recognition.","author":"Li Zechen","year":"2024","unstructured":"Zechen Li, Shohreh Deldari, Linyao Chen, Hao Xue, and Flora D Salim. 2024. Sensorllm: Aligning large language models with motion sensors for human activity recognition. (2024)."},{"key":"e_1_2_1_29_1","volume-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 6, 4, Article 170 (Jan.","author":"Liang Chen","year":"2023","unstructured":"Chen Liang, Chi Hsia, Chun Yu, Yukang Yan, Yuntao Wang, and Yuanchun Shi. 2023. DRG-Keyboard: Enabling Subtle Gesture Typing on the Fingertip with Dual IMU Rings. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 6, 4, Article 170 (Jan. 2023), 30 pages. https:\/\/doi.org\/10.1145\/3569463"},{"key":"e_1_2_1_30_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3478114","article-title":"DualRing: Enabling subtle and expressive hand interaction with dual IMU rings","volume":"5","author":"Liang Chen","year":"2021","unstructured":"Chen Liang, Chun Yu, Yue Qin, Yuntao Wang, and Yuanchun Shi. 2021. DualRing: Enabling subtle and expressive hand interaction with dual IMU rings. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 5, 3 (2021), 1\u201327.","journal-title":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/LED.2021.3078157"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.3390\/s18051545"},{"key":"e_1_2_1_33_1","first-page":"1","article-title":"Keep the Phone in Your Pocket","volume":"4","author":"Liu Guanhong","year":"2020","unstructured":"Guanhong Liu, Yizheng Gu, Yiwen Yin, Chun Yu, Yuntao Wang, Haipeng Mi, and Yuanchun Shi. 2020. Keep the Phone in Your Pocket. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4 (2020), 1 - 23. https:\/\/api.semanticscholar.org\/CorpusID:219688953","journal-title":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"},{"key":"e_1_2_1_34_1","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1007\/s41315-019-00085-4","article-title":"A new IMMU-based data glove for hand motion capture with optimized sensor layout","volume":"3","author":"Liu Quan","year":"2019","unstructured":"Quan Liu, Guoming Qian, Wei Meng, Qingsong Ai, Chaoyue Yin, and Zhengyu Fang. 2019. A new IMMU-based data glove for hand motion capture with optimized sensor layout. International Journal of Intelligent Robotics and Applications 3, 1 (2019), 19\u201332.","journal-title":"International Journal of Intelligent Robotics and Applications"},{"key":"e_1_2_1_35_1","volume-title":"Decoupled Weight Decay Regularization. arXiv preprint arXiv:1711.05101","author":"Loshchilov Ilya","year":"2017","unstructured":"Ilya Loshchilov and Frank Hutter. 2017. Decoupled Weight Decay Regularization. arXiv preprint arXiv:1711.05101 (2017)."},{"key":"e_1_2_1_36_1","volume-title":"SGDR: Stochastic Gradient Descent with Warm Restarts. In International Conference on Learning Representations.","author":"Loshchilov Ilya","year":"2017","unstructured":"Ilya Loshchilov and Frank Hutter. 2017. SGDR: Stochastic Gradient Descent with Warm Restarts. In International Conference on Learning Representations."},{"key":"e_1_2_1_37_1","volume-title":"European Conference on Computer Vision. Springer, 445\u2013465","author":"Ma Lingni","year":"2024","unstructured":"Lingni Ma, Yuting Ye, Fangzhou Hong, Vladimir Guzov, Yifeng Jiang, Rowan Postyeni, Luis Pesqueira, Alexander Gamino, Vijay Baiyya, Hyo Jin Kim, et al. 2024. Nymeria: A massive collection of multimodal egocentric daily motion in the wild. In European Conference on Computer Vision. Springer, 445\u2013465."},{"key":"e_1_2_1_38_1","unstructured":"Saif Mahmud Kian Mahmoodi Chi-Jung Lee Francois Guimbretiere and Cheng Zhang. 2024. WristSonic: Enabling Fine-grained Hand-Face Interactions on Smartwatches Using Active Acoustic Sensing. arXiv:2411.08217 [cs.HC] https:\/\/arxiv.org\/abs\/2411.08217"},{"key":"e_1_2_1_39_1","volume-title":"Automation","author":"Ma\u0144kowski Tomasz","year":"2017","unstructured":"Tomasz Ma\u0144kowski, Jakub Tomczy\u0144ski, and Piotr Kaczmarek. 2017. CIE-DataGlove, A Multi-IMU System for Hand Posture Tracking. In Automation 2017, Roman Szewczyk, Cezary Zieli\u0144ski, and Ma\u0142gorzata Kaliczy\u0144ska (Eds.). Springer International Publishing, Cham, 268\u2013276."},{"key":"e_1_2_1_40_1","volume-title":"Sensor-Based Hand Gesture Recognition Using One-Dimensional Deep Convolutional and Residual Bidirectional Gated Recurrent Unit Neural Network. Lobachevskii journal of mathematics 46, 1","author":"Mekruksavanich Sakorn","year":"2025","unstructured":"Sakorn Mekruksavanich, Wikanda Phaphan, and Anuchit Jitpattanakul. 2025. Sensor-Based Hand Gesture Recognition Using One-Dimensional Deep Convolutional and Residual Bidirectional Gated Recurrent Unit Neural Network. Lobachevskii journal of mathematics 46, 1 (2025), 464\u2013480."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2023.3235368"},{"key":"e_1_2_1_42_1","volume-title":"The 2023 Conference on Empirical Methods in Natural Language Processing.","author":"Moon Seungwhan","year":"2023","unstructured":"Seungwhan Moon, Andrea Madotto, Zhaojiang Lin, Aparajita Saraf, Amy L Bearman, and Babak Damavandi. 2023. IMU2CLIP: language-grounded motion sensor translation with multimodal contrastive learning. In The 2023 Conference on Empirical Methods in Natural Language Processing."},{"key":"e_1_2_1_43_1","doi-asserted-by":"crossref","first-page":"826","DOI":"10.1038\/s41597-023-02738-3","article-title":"Coming in handy: CeTI-Age\u2014A comprehensive database of kinematic hand movements across the lifespan","volume":"10","author":"Muschter Evelyn","year":"2023","unstructured":"Evelyn Muschter, Jonas Schulz, M\u00e1t\u00e9 T\u00f6m\u00f6sk\u00f6zi, Leonie Herbst, Lena K\u00fcssel, Merve Sefun\u00e7, Stefan Holtzhausen, Stefanie Speidel, Frank HP Fitzek, and Shu-Chen Li. 2023. Coming in handy: CeTI-Age\u2014A comprehensive database of kinematic hand movements across the lifespan. Scientific Data 10, 1 (2023), 826.","journal-title":"Scientific Data"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.3390\/app15041842"},{"key":"e_1_2_1_45_1","volume-title":"International conference on machine learning. PmLR, 8748\u20138763","author":"Radford Alec","year":"2021","unstructured":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning. PmLR, 8748\u20138763."},{"key":"e_1_2_1_46_1","volume-title":"Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct. 722\u2013731","author":"Ranjan Juhi","year":"2016","unstructured":"Juhi Ranjan and Kamin Whitehouse. 2016. Towards recognizing person-object interactions using a single wrist wearable device. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct. 722\u2013731."},{"key":"e_1_2_1_47_1","first-page":"3736","article-title":"Kalman and extended kalman filters: Concept, derivation and properties","volume":"43","author":"Ribeiro Maria Isabel","year":"2004","unstructured":"Maria Isabel Ribeiro. 2004. Kalman and extended kalman filters: Concept, derivation and properties. Institute for Systems and Robotics 43, 46 (2004), 3736\u20133741.","journal-title":"Institute for Systems and Robotics"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","unstructured":"Daniel Roggen Alberto Calatroni Long-Van Nguyen-Dinh Ricardo Chavarriaga and Hesam Sagha. 2010. OPPORTUNITY Activity Recognition. UCI Machine Learning Repository. DOI: https:\/\/doi.org\/10.24432\/C5M027.","DOI":"10.24432\/C5M027"},{"key":"e_1_2_1_49_1","doi-asserted-by":"crossref","first-page":"181955","DOI":"10.1109\/ACCESS.2020.3026965","article-title":"OREBA: A dataset for objectively recognizing eating behavior and associated intake","volume":"8","author":"Rouast Philipp V","year":"2020","unstructured":"Philipp V Rouast, Hamid Heydarian, Marc TP Adam, and Megan E Rollo. 2020. OREBA: A dataset for objectively recognizing eating behavior and associated intake. IEEE Access 8 (2020), 181955\u2013181963.","journal-title":"IEEE Access"},{"key":"e_1_2_1_50_1","volume-title":"Mojito: LLM-Aided Motion Instructor with Jitter-Reduced Inertial Tokens. arXiv preprint arXiv:2502.16175","author":"Shan Ziwei","year":"2025","unstructured":"Ziwei Shan, Yaoyu He, Chengfeng Zhao, Jiashen Du, Jingyan Zhang, Qixuan Zhang, Jingyi Yu, and Lan Xu. 2025. Mojito: LLM-Aided Motion Instructor with Jitter-Reduced Inertial Tokens. arXiv preprint arXiv:2502.16175 (2025)."},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2024.3456179"},{"key":"e_1_2_1_52_1","first-page":"6531","article-title":"Motion transformer with global intention localization and local movement refinement","volume":"35","author":"Shi Shaoshuai","year":"2022","unstructured":"Shaoshuai Shi, Li Jiang, Dengxin Dai, and Bernt Schiele. 2022. Motion transformer with global intention localization and local movement refinement. Advances in Neural Information Processing Systems 35 (2022), 6531\u20136543.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_53_1","volume-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 2, Article 59 (June","author":"Shi Yilei","year":"2020","unstructured":"Yilei Shi, Haimo Zhang, Kaixing Zhao, Jiashuo Cao, Mengmeng Sun, and Suranga Nanayakkara. 2020. Ready, Steady, Touch! Sensing Physical Contact with a Finger-Mounted IMU. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 2, Article 59 (June 2020), 25 pages. https:\/\/doi.org\/10.1145\/3397309"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0227039"},{"key":"e_1_2_1_55_1","doi-asserted-by":"crossref","first-page":"692","DOI":"10.26599\/TST.2021.9010017","article-title":"Inertial motion tracking on mobile and wearable devices: Recent advancements and challenges","volume":"26","author":"Song Zhipeng","year":"2021","unstructured":"Zhipeng Song, Zhichao Cao, Zhenjiang Li, Jiliang Wang, and Yunhao Liu. 2021. Inertial motion tracking on mobile and wearable devices: Recent advancements and challenges. Tsinghua Science and Technology 26, 5 (2021), 692\u2013705.","journal-title":"Tsinghua Science and Technology"},{"key":"e_1_2_1_56_1","volume-title":"Convolutional Neural Network Array for Sign Language Recognition Using Wearable IMUs. In 2019 6th International Conference on Signal Processing and Integrated Networks (SPIN). 483\u2013488","author":"Suri Karush","year":"2019","unstructured":"Karush Suri and Rinki Gupta. 2019. Convolutional Neural Network Array for Sign Language Recognition Using Wearable IMUs. In 2019 6th International Conference on Signal Processing and Integrated Networks (SPIN). 483\u2013488. https:\/\/doi.org\/10.1109\/SPIN.2019.8711745"},{"key":"e_1_2_1_57_1","doi-asserted-by":"crossref","first-page":"41","DOI":"10.3390\/jlpea12030041","article-title":"Electrical impedance tomography for hand gesture recognition for hmi interaction applications","volume":"12","author":"Vaquero-Gallardo Noelia","year":"2022","unstructured":"Noelia Vaquero-Gallardo and Herminio Mart\u00ednez-Garc\u00eda. 2022. Electrical impedance tomography for hand gesture recognition for hmi interaction applications. Journal of Low Power Electronics and Applications 12, 3 (2022), 41.","journal-title":"Journal of Low Power Electronics and Applications"},{"key":"e_1_2_1_58_1","volume-title":"Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1\u201318","author":"Waghmare Anandghan","year":"2023","unstructured":"Anandghan Waghmare, Youssef Ben Taleb, Ishan Chatterjee, Arjun Narendra, and Shwetak Patel. 2023. Z-ring: Single-point bio-impedance sensing for gesture, touch, object and user recognition. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1\u201318."},{"key":"e_1_2_1_59_1","volume-title":"Proceedings of the Computer Vision and Pattern Recognition Conference. 22668\u201322679","author":"Wang Jian","year":"2025","unstructured":"Jian Wang, Rishabh Dabral, Diogo Luvizon, Zhe Cao, Lingjie Liu, Thabo Beeler, and Christian Theobalt. 2025. Ego4o: Egocentric Human Motion Capture and Understanding from Multi-Modal Input. In Proceedings of the Computer Vision and Pattern Recognition Conference. 22668\u201322679."},{"key":"e_1_2_1_60_1","volume-title":"Motiongpt-2: A general-purpose motion-language model for motion generation and understanding. arXiv preprint arXiv:2410.21747","author":"Wang Yuan","year":"2024","unstructured":"Yuan Wang, Di Huang, Yaqi Zhang, Wanli Ouyang, Jile Jiao, Xuetao Feng, Yan Zhou, Pengfei Wan, Shixiang Tang, and Dan Xu. 2024. Motiongpt-2: A general-purpose motion-language model for motion generation and understanding. arXiv preprint arXiv:2410.21747 (2024)."},{"key":"e_1_2_1_61_1","volume-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 9, 3, Article 139 (Sept.","author":"Wei Qingxin","year":"2025","unstructured":"Qingxin Wei, Jiaming Huang, Yi Gao, and Wei Dong. 2025. One Model to Fit Them All: Universal IMU-based Human Activity Recognition with LLM-assisted Cross-dataset Representation. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 9, 3, Article 139 (Sept. 2025), 22 pages. https:\/\/doi.org\/10.1145\/3749509"},{"key":"e_1_2_1_62_1","volume-title":"Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology. 1\u201316","author":"Xiao Yingjing","year":"2025","unstructured":"Yingjing Xiao, Zhichao Huang, Junbin Ren, Yuting Bai, Haichuan Song, Zhanpeng Jin, and Yang Gao. 2025. Wrist2Finger: Sensing Fingertip Force for Force-Aware Hand Interaction with a Ring-Watch Wearable. In Proceedings of the 38th Annual ACM Symposium on User Interface Software and Technology. 1\u201316."},{"key":"e_1_2_1_63_1","volume-title":"Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems. 220\u2013233","author":"Xu Huatao","year":"2021","unstructured":"Huatao Xu, Pengfei Zhou, Rui Tan, Mo Li, and Guobin Shen. 2021. Limu-bert: Unleashing the potential of unlabeled data for imu sensing applications. In Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems. 220\u2013233."},{"key":"e_1_2_1_64_1","volume-title":"Proceedings of the 2022 International Conference on Multimodal Interaction. 597\u2013604","author":"Yang Liu","year":"2022","unstructured":"Liu Yang, Catherine Achard, and Catherine Pelachaud. 2022. Multimodal classification of interruptions in humans' interaction. In Proceedings of the 2022 International Conference on Multimodal Interaction. 597\u2013604."},{"key":"e_1_2_1_65_1","volume-title":"Llasa: Scaling train-time and inference-time compute for llama-based speech synthesis. arXiv preprint arXiv:2502.04128","author":"Ye Zhen","year":"2025","unstructured":"Zhen Ye, Xinfa Zhu, Chi-Min Chan, Xinsheng Wang, Xu Tan, Jiahe Lei, Yi Peng, Haohe Liu, Yizhu Jin, Zheqi Dai, et al. 2025. Llasa: Scaling train-time and inference-time compute for llama-based speech synthesis. arXiv preprint arXiv:2502.04128 (2025)."},{"key":"e_1_2_1_66_1","first-page":"1","article-title":"Ring-a-Pose: A Ring for Continuous Hand Pose Tracking","volume":"8","author":"Yu Tianhong Catherine","year":"2024","unstructured":"Tianhong Catherine Yu, Guilin Hu, Ruidong Zhang, Hyunchul Lim, Saif Mahmud, Chi-Jung Lee, Ke Li, Devansh Agarwal, Shuyang Nie, Jinseok Oh, et al. 2024. Ring-a-Pose: A Ring for Continuous Hand Pose Tracking. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8, 4 (2024), 1\u201330.","journal-title":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"},{"key":"e_1_2_1_67_1","first-page":"107469","article-title":"UniMTS: Unified Pre-training for Motion Time Series","volume":"37","author":"Zhang Xiyuan","year":"2024","unstructured":"Xiyuan Zhang, Diyan Teng, Ranak Roy Chowdhury, Shuheng Li, Dezhi Hong, Rajesh Gupta, and Jingbo Shang. 2024. UniMTS: Unified Pre-training for Motion Time Series. Advances in Neural Information Processing Systems 37 (2024), 107469\u2013107493.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.3390\/s21175713"},{"key":"e_1_2_1_69_1","first-page":"11106","volume-title":"Proceedings of the AAAI conference on artificial intelligence","volume":"35","author":"Zhou Haoyi","year":"2021","unstructured":"Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. 2021. Informer: Beyond efficient transformer for long sequence time-series forecasting. In Proceedings of the AAAI conference on artificial intelligence, Vol. 35.11106-11115."},{"key":"e_1_2_1_70_1","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision. 22968\u201322977","author":"Zhu Yongxin","year":"2025","unstructured":"Yongxin Zhu, Bocheng Li, Yifei Xin, Zhihua Xia, and Linli Xu. 2025. Addressing representation collapse in vector quantized models with one linear layer. In Proceedings of the IEEE\/CVF International Conference on Computer Vision. 22968\u201322977."},{"key":"e_1_2_1_71_1","volume-title":"The 34th Annual ACM Symposium on User Interface Software and Technology","author":"Zhuang Yuzhou","year":"2021","unstructured":"Yuzhou Zhuang, Yuntao Wang, Yukang Yan, Xuhai Xu, and Yuanchun Shi. 2021. ReflecTrack: Enabling 3D Acoustic Position Tracking Using Commodity Dual-Microphone Smartphones. In The 34th Annual ACM Symposium on User Interface Software and Technology (Virtual Event, USA) (UIST '21). Association for Computing Machinery, New York, NY, USA, 1050\u20131062. https:\/\/doi.org\/10.1145\/3472749.3474805"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3810222","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,15]],"date-time":"2026-06-15T17:09:20Z","timestamp":1781543360000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3810222"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,15]]},"references-count":71,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,6,15]]}},"alternative-id":["10.1145\/3810222"],"URL":"https:\/\/doi.org\/10.1145\/3810222","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,15]]},"assertion":[{"value":"2026-06-15","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}