{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T19:52:22Z","timestamp":1764705142740,"version":"3.46.0"},"reference-count":61,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T00:00:00Z","timestamp":1764633600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"NSF","award":["2246080, 2427915, 2345057"],"award-info":[{"award-number":["2246080, 2427915, 2345057"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2025,12,2]]},"abstract":"<jats:p>Human Activity Recognition (HAR) using Inertial Measurement Unit (IMU) sensors is critical for applications in healthcare, safety, and industrial production. However, variations in activity patterns, device types, and sensor placements create distribution gaps across datasets, reducing the performance of HAR models. To address this, we propose LanHAR, a novel system that leverages Large Language Models (LLMs) to generate semantic interpretations of sensor readings and activity labels for cross-dataset HAR. This approach not only mitigates cross-dataset heterogeneity but also enhances the recognition of new activities. LanHAR employs an iterative re-generation method to produce high-quality semantic interpretations with LLMs and a two-stage training framework that bridges the semantic interpretations of sensor readings and activity labels. This ultimately leads to a lightweight sensor encoder suitable for mobile deployment, enabling any sensor reading to be mapped into the semantic interpretation space. Experiments on five public datasets demonstrate that our approach significantly outperforms state-of-the-art methods in both cross-dataset HAR and new activity recognition. The source code is publicly available at https:\/\/github.com\/DASHLab\/LanHAR.<\/jats:p>","DOI":"10.1145\/3770652","type":"journal-article","created":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T19:42:32Z","timestamp":1764704552000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Large Language Model-guided Semantic Alignment for Human Activity Recognition"],"prefix":"10.1145","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-5967-0194","authenticated-orcid":false,"given":"Hua","family":"Yan","sequence":"first","affiliation":[{"name":"Lehigh University, Bethlehem, Pennsylvania, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1864-6761","authenticated-orcid":false,"given":"Heng","family":"Tan","sequence":"additional","affiliation":[{"name":"Lehigh University, Bethlehem, Pennsylvania, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1226-341X","authenticated-orcid":false,"given":"Yi","family":"Ding","sequence":"additional","affiliation":[{"name":"Computer Science, University of Texas at Dallas, Richardson, Texas, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1836-1122","authenticated-orcid":false,"given":"Pengfei","family":"Zhou","sequence":"additional","affiliation":[{"name":"School of Computing and Information, University of Pittsburgh, Pittsburgh, Pennsylvania, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0705-6990","authenticated-orcid":false,"given":"Vinod","family":"Namboodiri","sequence":"additional","affiliation":[{"name":"Lehigh University, Bethlehem, Pennsylvania, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1627-5503","authenticated-orcid":false,"given":"Yu","family":"Yang","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Lehigh University, Bethlehem, Pennsylvania, USA"}]}],"member":"320","published-online":{"date-parts":[[2025,12,2]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"2024 IEEE International Conference on Smart Computing (SMARTCOMP). IEEE, 55\u201362","author":"Arrotta Luca","year":"2024","unstructured":"Luca Arrotta, Claudio Bettini, Gabriele Civitarese, and Michele Fiori. 2024. Contextgpt: Infusing llms knowledge into neuro-symbolic activity recognition models. In 2024 IEEE International Conference on Smart Computing (SMARTCOMP). IEEE, 55\u201362."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11063-021-10562-2"},{"key":"e_1_2_1_3_1","volume-title":"2019 international conference on electrical, computer and communication engineering (ecce). IEEE, 1\u20136.","author":"Barna Arnab","year":"2019","unstructured":"Arnab Barna, Abdul Kadar Muhammad Masum, Mohammad Emdad Hossain, Erfanul Hoque Bahadur, and Mohammed Shamsul Alam. 2019. A study on human activity recognition using gyroscope, accelerometer, temperature and humidity data. In 2019 international conference on electrical, computer and communication engineering (ecce). IEEE, 1\u20136."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3380985"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.3390\/s21030692"},{"key":"e_1_2_1_6_1","volume-title":"Nuactiv: Recognizing unseen new activities using semantic attribute-based learning. In Proceeding of the 11th annual international conference on Mobile systems, applications, and services. 361\u2013374.","author":"Cheng Heng-Tze","year":"2013","unstructured":"Heng-Tze Cheng, Feng-Tso Sun, Martin Griss, Paul Davis, Jianguo Li, and Di You. 2013. Nuactiv: Recognizing unseen new activities using semantic attribute-based learning. In Proceeding of the 11th annual international conference on Mobile systems, applications, and services. 361\u2013374."},{"key":"e_1_2_1_7_1","volume-title":"Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805","author":"Devlin Jacob","year":"2018","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)."},{"key":"e_1_2_1_8_1","volume-title":"Lightweight transformers for human activity recognition on mobile devices. arXiv preprint arXiv:2209.11750","author":"Ek Sannara","year":"2022","unstructured":"Sannara Ek, Fran\u00e7ois Portet, and Philippe Lalanda. 2022. Lightweight transformers for human activity recognition on mobile devices. arXiv preprint arXiv:2209.11750 (2022)."},{"key":"e_1_2_1_9_1","volume-title":"HARGPT: Are LLMs Zero-Shot Human Activity Recognizers? arXiv preprint arXiv:2403.02727","author":"Ji Sijie","year":"2024","unstructured":"Sijie Ji, Xinzhe Zheng, and Chenshu Wu. 2024. HARGPT: Are LLMs Zero-Shot Human Activity Recognizers? arXiv preprint arXiv:2403.02727 (2024)."},{"key":"e_1_2_1_10_1","first-page":"20067","article-title":"Motiongpt: Human motion as a foreign language","volume":"36","author":"Jiang Biao","year":"2023","unstructured":"Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, and Tao Chen. 2023. Motiongpt: Human motion as a foreign language. Advances in Neural Information Processing Systems 36 (2023), 20067\u201320079.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_11_1","volume-title":"Motiongpt: Human motion as a foreign language. Advances in Neural Information Processing Systems 36","author":"Jiang Biao","year":"2024","unstructured":"Biao Jiang, Xin Chen, Wen Liu, Jingyi Yu, Gang Yu, and Tao Chen. 2024. Motiongpt: Human motion as a foreign language. Advances in Neural Information Processing Systems 36 (2024)."},{"key":"e_1_2_1_12_1","volume-title":"Time-llm: Time series forecasting by reprogramming large language models. arXiv preprint arXiv:2310.01728","author":"Jin Ming","year":"2023","unstructured":"Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Y Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, et al. 2023. Time-llm: Time series forecasting by reprogramming large language models. arXiv preprint arXiv:2310.01728 (2023)."},{"key":"e_1_2_1_13_1","volume-title":"Accelerometry: a technique for quantifying movement patterns during walking. Gait & posture 28, 1","author":"Kavanagh Justin J","year":"2008","unstructured":"Justin J Kavanagh and Hylton B Menz. 2008. Accelerometry: a technique for quantifying movement patterns during walking. Gait & posture 28, 1 (2008), 1\u201315."},{"key":"e_1_2_1_14_1","volume-title":"2018 IEEE international conference on pervasive computing and communications (PerCom). IEEE, 1\u20139.","author":"Al Hafiz Khan Md Abdullah","year":"2018","unstructured":"Md Abdullah Al Hafiz Khan, Nirmalya Roy, and Archan Misra. 2018. Scaling human activity recognition via deep learning-based domain adaptation. In 2018 IEEE international conference on pervasive computing and communications (PerCom). IEEE, 1\u20139."},{"key":"e_1_2_1_15_1","volume-title":"Health-LLM: large language models for health prediction via wearable sensor data, arXiv. arXiv preprint arXiv:2401.06866","author":"Kim Y","year":"2024","unstructured":"Y Kim, X Xu, D McDuff, C Breazeal, and HW Park. 2024. Health-LLM: large language models for health prediction via wearable sensor data, arXiv. arXiv preprint arXiv:2401.06866 (2024)."},{"key":"e_1_2_1_16_1","volume-title":"On information and sufficiency. The annals of mathematical statistics 22, 1","author":"Kullback Solomon","year":"1951","unstructured":"Solomon Kullback and Richard A Leibler. 1951. On information and sufficiency. The annals of mathematical statistics 22, 1 (1951), 79\u201386."},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the 30th Annual International Conference on Mobile Computing and Networking. 1119\u20131133","author":"Lee Sunjae","year":"2024","unstructured":"Sunjae Lee, Junyoung Choi, Jungjae Lee, Munim Hasan Wasi, Hojun Choi, Steve Ko, Sangeun Oh, and Insik Shin. 2024. Mobilegpt: Augmenting llm with human-like app memory for mobile task automation. In Proceedings of the 30th Annual International Conference on Mobile Computing and Networking. 1119\u20131133."},{"key":"e_1_2_1_18_1","volume-title":"Language-based cross modality transfer for sensor-based human activity recognition. arXiv preprint arXiv:2402.01049","author":"Leng Zikang","year":"2024","unstructured":"Zikang Leng, Amitrajit Bhattacharjee, Hrudhai Rajasekhar, Lizhe Zhang, Elizabeth Bruda, Hyeokhyen Kwon, and Thomas Pl\u00f6tz. 2024. Imugpt 2.0: Language-based cross modality transfer for sensor-based human activity recognition. arXiv preprint arXiv:2402.01049 (2024)."},{"key":"e_1_2_1_19_1","volume-title":"fast fall detection using gyroscopes and accelerometer-derived posture information. In 2009 sixth international workshop on wearable and implantable body sensor networks","author":"Li Qiang","unstructured":"Qiang Li, John A Stankovic, Mark A Hanson, Adam T Barth, John Lach, and Gang Zhou. 2009. Accurate, fast fall detection using gyroscopes and accelerometer-derived posture information. In 2009 sixth international workshop on wearable and implantable body sensor networks. IEEE, 138\u2013143."},{"key":"e_1_2_1_20_1","volume-title":"Computer Networks and Social Media Conference (SEEDA-CECNSM). IEEE, 1\u20136.","author":"Liagkou Vasiliki","year":"2022","unstructured":"Vasiliki Liagkou, Sofia Sakka, and Chrysostomos Stylios. 2022. Security and Privacy Vulnerabilities in Human Activity Recognition systems. In 2022 7th South-East Europe Design Automation, Computer Engineering, Computer Networks and Social Media Conference (SEEDA-CECNSM). IEEE, 1\u20136."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3534678.3539167"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3534589"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3302505.3310068"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3699736"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3744746"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.3390\/s22010134"},{"key":"e_1_2_1_27_1","volume-title":"Proceedings of the 1st International Conference on Electronics, Biomedical Engineering, and Health Informatics: ICEBEHI","author":"Nurwulan Nurul Retno","year":"2020","unstructured":"Nurul Retno Nurwulan and Gjergji Selamaj. 2021. A comparative evaluation of acceleration and jerk in human activity recognition using machine learning techniques. In Proceedings of the 1st International Conference on Electronics, Biomedical Engineering, and Health Informatics: ICEBEHI 2020, 8\u20139 October, Surabaya, Indonesia. Springer, 55\u201361."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.3390\/s16010115"},{"key":"e_1_2_1_29_1","volume-title":"2023 IEEE International Conference on Smart Computing (SMARTCOMP). IEEE, 33\u201340","author":"Presotto Riccardo","year":"2023","unstructured":"Riccardo Presotto, Sannara Ek, Gabriele Civitarese, Fran\u00e7ois Portet, Philippe Lalanda, and Claudio Bettini. 2023. Combining public human activity recognition datasets to mitigate labeled data scarcity. In 2023 IEEE International Conference on Smart Computing (SMARTCOMP). IEEE, 33\u201340."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3534678.3539134"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3369818"},{"key":"e_1_2_1_32_1","volume-title":"International conference on machine learning. PMLR, 8748\u20138763","author":"Radford Alec","year":"2021","unstructured":"Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning. PMLR, 8748\u20138763."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","unstructured":"Attila Reiss. 2012. PAMAP2 Physical Activity Monitoring. UCI Machine Learning Repository. DOI: https:\/\/doi.org\/10.24432\/C5NW2H.","DOI":"10.24432\/C5NW2H"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2015.07.085"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3328932"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.3390\/s140610146"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2809695.2809718"},{"key":"e_1_2_1_38_1","volume-title":"Innovation in health informatics","author":"Subasi Abdulhamit","unstructured":"Abdulhamit Subasi, Kholoud Khateeb, Tayeb Brahimi, and Akila Sarirete. 2020. Human activity recognition using machine learning methods in a smart healthcare environment. In Innovation in health informatics. Elsevier, 123\u2013144."},{"key":"e_1_2_1_39_1","volume-title":"IoT based mobile healthcare system for human activity recognition. In 2018 15th learning and technology conference (L&T)","author":"Subasi Abdulhamit","unstructured":"Abdulhamit Subasi, Mariam Radhwan, Rabea Kurdi, and Kholoud Khateeb. 2018. IoT based mobile healthcare system for human activity recognition. In 2018 15th learning and technology conference (L&T). IEEE, 29\u201334."},{"key":"e_1_2_1_40_1","volume-title":"2022 IEEE 16th International Symposium on Medical Information and Communication Technology (ISMICT). IEEE, 1\u20136.","author":"Sun Han","year":"2022","unstructured":"Han Sun and Yu Chen. 2022. Real-time elderly monitoring for senior safety by lightweight human action recognition. In 2022 IEEE 16th International Symposium on Medical Information and Communication Technology (ISMICT). IEEE, 1\u20136."},{"key":"e_1_2_1_41_1","volume-title":"LLM-Guided Reinforcement Learning: Addressing Training Bottlenecks through Policy Modulation. arXiv preprint arXiv:2505.20671","author":"Tan Heng","year":"2025","unstructured":"Heng Tan, Hua Yan, and Yu Yang. 2025. LLM-Guided Reinforcement Learning: Addressing Training Bottlenecks through Policy Modulation. arXiv preprint arXiv:2505.20671 (2025)."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448112"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3136755.3136817"},{"key":"e_1_2_1_44_1","article-title":"Visualizing data using t-SNE","volume":"9","author":"der Maaten Laurens Van","year":"2008","unstructured":"Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, 11 (2008).","journal-title":"Journal of machine learning research"},{"key":"e_1_2_1_45_1","volume-title":"Attention is all you need. Advances in Neural Information Processing Systems","author":"Vaswani A","year":"2017","unstructured":"A Vaswani. 2017. Attention is all you need. Advances in Neural Information Processing Systems (2017)."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSEN.2016.2545708"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.proeng.2011.08.331"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSEN.2022.3214198"},{"key":"e_1_2_1_49_1","volume-title":"Denny Zhou, et al.","author":"Wei Jason","year":"2022","unstructured":"Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35 (2022), 24824\u201324837."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3636534.3649379"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3638550.3641130"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3570361.3613299"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3485730.3485937"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3589334.3645595"},{"key":"e_1_2_1_55_1","volume-title":"Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. 2877\u20132886","author":"Yan Hua","year":"2023","unstructured":"Hua Yan, Hao Wang, Desheng Zhang, and Yu Yang. 2023. Identifying regional driving risks via transductive cross-city transfer learning under negative transfer. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. 2877\u20132886."},{"key":"e_1_2_1_56_1","volume-title":"Ijcai","volume":"15","author":"Yang Jianbo","year":"2015","unstructured":"Jianbo Yang, Minh Nhut Nguyen, Phyo Phyo San, Xiaoli Li, and Shonali Krishnaswamy. 2015. Deep convolutional neural networks on multichannel time series for human activity recognition. In Ijcai, Vol. 15. Buenos Aires, Argentina, 3995\u20134001."},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/3038912.3052577"},{"key":"e_1_2_1_58_1","volume-title":"Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. 3340\u20133350","author":"Zhang Xiyuan","year":"2023","unstructured":"Xiyuan Zhang, Ranak Roy Chowdhury, Jiayun Zhang, Dezhi Hong, Rajesh K Gupta, and Jingbo Shang. 2023. Unleashing the Power of Shared Label Structures for Human Activity Recognition. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management. 3340\u20133350."},{"key":"e_1_2_1_59_1","volume-title":"Shuheng Li, Dezhi Hong, Rajesh K. Gupta, and Jingbo Shang.","author":"Zhang Xiyuan","year":"2024","unstructured":"Xiyuan Zhang, Diyan Teng, Ranak Roy Chowdhury, Shuheng Li, Dezhi Hong, Rajesh K. Gupta, and Jingbo Shang. 2024. UniMTS: Unified Pre-training for Motion Time Series. arXiv:2410.19818 [eess.SP] https:\/\/arxiv.org\/abs\/2410.19818"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.3390\/s18072146"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/SECON48991.2020.9158431"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3770652","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3770652","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T19:49:24Z","timestamp":1764704964000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3770652"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,2]]},"references-count":61,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,12,2]]}},"alternative-id":["10.1145\/3770652"],"URL":"https:\/\/doi.org\/10.1145\/3770652","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,2]]},"assertion":[{"value":"2025-12-02","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}