{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T19:51:58Z","timestamp":1764705118240,"version":"3.46.0"},"reference-count":49,"publisher":"Association for Computing Machinery (ACM)","issue":"4","funder":[{"DOI":"10.13039\/501100001809","name":"National Nature Science Foundation of China","doi-asserted-by":"crossref","award":["62472387,62471438,U24A20242"],"award-info":[{"award-number":["62472387,62471438,U24A20242"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"National Key Research and Development Program Project of China","award":["2024YFC3306902"],"award-info":[{"award-number":["2024YFC3306902"]}]},{"name":"Zhejiang Provincial Natural Science Foundation of Major Program","award":["LDQ24F020001"],"award-info":[{"award-number":["LDQ24F020001"]}]},{"DOI":"10.13039\/501100004731","name":"Zhejiang Provincial Natural Science Foundation","doi-asserted-by":"crossref","award":["LDT23F02024F02"],"award-info":[{"award-number":["LDT23F02024F02"]}],"id":[{"id":"10.13039\/501100004731","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2025,12,2]]},"abstract":"<jats:p>\n                    Wearable-based human activity recognition (HAR) typically uses motion sensor data, such as inertial measurement unit (IMU) signals, to identify human movements. While effective in controlled scenarios, traditional HAR models are trained on a fixed set of activities and fail to generalize to new or unseen actions. This limitation motivates the use of zero-shot learning (ZSL), which aims to recognize unseen activities without direct training examples. Existing ZSL methods often rely on projecting seen and unseen classes into a shared latent space using external semantic information, such as visual or textual data. However, visual data are commonly unavailable in wearable settings, and text-based semantics from activity labels or coarse descriptions lack the detail needed for accurate recognition. Recent work explores large language models (LLMs) to provide prior knowledge through question-answering mechanisms. While promising, these approaches do not use raw sensor data directly and often miss important contextual signals. We propose\n                    <jats:italic toggle=\"yes\">IMUZero<\/jats:italic>\n                    , a ZSL framework that fuses sensor signals with LLM-generated semantic attributes. Our method uses LLMs to produce fine-grained, decomposable activity attributes without additional LLM-based training, preserving sensor context. We also introduce a channel shuffle order constraint that models axial bias to improve generalization. Experiments on four public datasets show that our method outperforms existing ZSL approaches that rely on learned semantic embeddings. We release the code at https:\/\/github.com\/Was-Lab\/IMUZero.\n                  <\/jats:p>","DOI":"10.1145\/3770669","type":"journal-article","created":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T19:42:32Z","timestamp":1764704552000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["<i>IMUZero:<\/i>\n                    Zero-Shot Human Activity Recognition by Language-Based Cross Modality Fusion"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1427-1253","authenticated-orcid":false,"given":"Jie","family":"Su","sequence":"first","affiliation":[{"name":"Zhejiang University of Technology, Hangzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-4020-4108","authenticated-orcid":false,"given":"Fengtong","family":"Ge","sequence":"additional","affiliation":[{"name":"Zhejiang University of Technology, Hangzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2914-912X","authenticated-orcid":false,"given":"Zhenyu","family":"Wen","sequence":"additional","affiliation":[{"name":"Zhejiang University of Technology, Hangzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5946-9434","authenticated-orcid":false,"given":"Taotao","family":"Li","sequence":"additional","affiliation":[{"name":"Zhejiang University of Technology, Hangzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3324-0591","authenticated-orcid":false,"given":"Yang","family":"Bai","sequence":"additional","affiliation":[{"name":"Agency for Science, Technology and Research (A*STAR), Singapore, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0152-4063","authenticated-orcid":false,"given":"Yejian","family":"Zhou","sequence":"additional","affiliation":[{"name":"Zhejiang University of Technology, Hangzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0958-7285","authenticated-orcid":false,"given":"Xiaoqin","family":"Zhang","sequence":"additional","affiliation":[{"name":"Zhejiang University of Technology, Hangzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,12,2]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-24646-6_1"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2021\/597"},{"key":"e_1_2_1_3_1","volume-title":"Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs","author":"Chen Liang-Chieh","year":"2017","unstructured":"Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. 2017. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence 40, 4 (2017), 834\u2013848."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v36i1.19909"},{"key":"e_1_2_1_5_1","first-page":"16622","article-title":"Hsva: Hierarchical semantic-visual adaptation for zero-shot learning","volume":"34","author":"Chen Shiming","year":"2021","unstructured":"Shiming Chen, Guosen Xie, Yang Liu, Qinmu Peng, Baigui Sun, Hao Li, Xinge You, and Ling Shao. 2021. Hsva: Hierarchical semantic-visual adaptation for zero-shot learning. Advances in Neural Information Processing Systems 34 (2021), 16622\u201316634.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2022.109270"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2493432.2493511"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2493432.2493511"},{"key":"e_1_2_1_9_1","volume-title":"Fast Fourier transforms: a tutorial review and a state of the art. Signal processing 19, 4","author":"Duhamel Pierre","year":"1990","unstructured":"Pierre Duhamel and Martin Vetterli. 1990. Fast Fourier transforms: a tutorial review and a state of the art. Signal processing 19, 4 (1990), 259\u2013299."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3314399"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3090076"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2971648.2971708"},{"key":"e_1_2_1_13_1","volume-title":"convolutional, and recurrent models for human activity recognition using wearables. arXiv preprint arXiv:1604.08880","author":"Hammerla Nils Y","year":"2016","unstructured":"Nils Y Hammerla, Shane Halloran, and Thomas Pl\u00f6tz. 2016. Deep, convolutional, and recurrent models for human activity recognition using wearables. arXiv preprint arXiv:1604.08880 (2016)."},{"key":"e_1_2_1_14_1","volume-title":"Long short-term memory. Neural computation 9, 8","author":"Hochreiter Sepp","year":"1997","unstructured":"Sepp Hochreiter and J\u00fcrgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735\u20131780."},{"key":"e_1_2_1_15_1","first-page":"19849","article-title":"Compositional zero-shot learning via fine-grained dense feature composition","volume":"33","author":"Huynh Dat","year":"2020","unstructured":"Dat Huynh and Ehsan Elhamifar. 2020. Compositional zero-shot learning via fine-grained dense feature composition. Advances in Neural Information Processing Systems 33 (2020), 19849\u201319860.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00454"},{"key":"e_1_2_1_17_1","volume-title":"HARGPT: Are LLMs Zero-Shot Human Activity Recognizers? arXiv preprint arXiv:2403.02727","author":"Ji Sijie","year":"2024","unstructured":"Sijie Ji, Xinzhe Zheng, and Chenshu Wu. 2024. HARGPT: Are LLMs Zero-Shot Human Activity Recognizers? arXiv preprint arXiv:2403.02727 (2024)."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.453"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130927"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1964897.1964918"},{"key":"e_1_2_1_21_1","doi-asserted-by":"crossref","unstructured":"Christoph H Lampert Hannes Nickisch and Stefan Harmeling. 2009. Learning to detect unseen object classes by between-class attribute transfer. In CVPR.","DOI":"10.1109\/CVPRW.2009.5206594"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3678545"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2023.3294505"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.3390\/s19225043"},{"key":"e_1_2_1_25_1","volume-title":"Imu2clip: Multimodal contrastive learning for imu motion sensors from egocentric videos and text. arXiv preprint arXiv:2210.14395","author":"Moon Seungwhan","year":"2022","unstructured":"Seungwhan Moon, Andrea Madotto, Zhaojiang Lin, Alireza Dirafzoon, Aparajita Saraf, Amy Bearman, and Babak Damavandi. 2022. Imu2clip: Multimodal contrastive learning for imu motion sensors from egocentric videos and text. arXiv preprint arXiv:2210.14395 (2022)."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3267242.3267287"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.3390\/s16010115"},{"key":"e_1_2_1_28_1","volume-title":"Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 32","author":"Paszke Adam","year":"2019","unstructured":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 32 (2019)."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2370216.2370276"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2012.256"},{"key":"e_1_2_1_31_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3596234","article-title":"ConvBoost: Boosting ConvNets for sensor-based activity recognition","volume":"7","author":"Shao Shuai","year":"2023","unstructured":"Shuai Shao, Yu Guan, Bing Zhai, Paolo Missier, and Thomas Pl\u00f6tz. 2023. ConvBoost: Boosting ConvNets for sensor-based activity recognition. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 7, 2 (2023), 1\u201321.","journal-title":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00113"},{"key":"e_1_2_1_33_1","unstructured":"Jie Su Peng Sun Yuting Jiang Zhenyu Wen Fangda Guo Yiming Wu Zhen Hong Haoran Duan Yawen Huang Rajiv Ranjan et al. 2024. A semantic-consistent few-shot modulation recognition framework for IoT applications. IEEE Transactions on Neural Networks and Learning Systems (2024)."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3517252"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3494995"},{"key":"e_1_2_1_36_1","volume-title":"Attention is all you need. Advances in Neural Information Processing Systems","author":"Vaswani A","year":"2017","unstructured":"A Vaswani. 2017. Attention is all you need. Advances in Neural Information Processing Systems (2017)."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3582690"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3106426.3106526"},{"key":"e_1_2_1_39_1","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision. 3032\u20133042","author":"Wang Zhengbo","year":"2023","unstructured":"Zhengbo Wang, Jian Liang, Ran He, Nan Xu, Zilei Wang, and Tieniu Tan. 2023. Improving zero-shot generalization for clip with synthesized prompts. In Proceedings of the IEEE\/CVF International Conference on Computer Vision. 3032\u20133042."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10489-023-04808-w"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-47426-3_17"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01052"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3570361.3613299"},{"key":"e_1_2_1_44_1","volume-title":"Ijcai","volume":"15","author":"Yang Jianbo","year":"2015","unstructured":"Jianbo Yang, Minh Nhut Nguyen, Phyo Phyo San, Xiaoli Li, and Shonali Krishnaswamy. 2015. Deep convolutional neural networks on multichannel time series for human activity recognition. In Ijcai, Vol. 15. Buenos Aires, Argentina, 3995\u20134001."},{"key":"e_1_2_1_45_1","unstructured":"Vladimir M Zatsiorsky. 2002. Kinetics of human motion. Human kinetics."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3397325"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3637528.3671828"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01249-6_17"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3659589"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3770669","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T19:47:13Z","timestamp":1764704833000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3770669"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,2]]},"references-count":49,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,12,2]]}},"alternative-id":["10.1145\/3770669"],"URL":"https:\/\/doi.org\/10.1145\/3770669","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,2]]},"assertion":[{"value":"2025-12-02","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}