{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T16:45:56Z","timestamp":1776444356803,"version":"3.51.2"},"reference-count":52,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2023,12,19]],"date-time":"2023-12-19T00:00:00Z","timestamp":1702944000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100006374","name":"Defense Advanced Research Projects Agency","doi-asserted-by":"publisher","award":["HR00111780032-WASH-FP-031"],"award-info":[{"award-number":["HR00111780032-WASH-FP-031"]}],"id":[{"id":"10.13039\/501100006374","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2023,12,19]]},"abstract":"<jats:p>Human Activity Recognition (HAR) is a challenging, multi-label classification problem as activities may co-occur and sensor signals corresponding to the same activity may vary in different contexts (e.g., different device placements). This paper proposes a Deep Heterogeneous Contrastive Hyper-Graph Learning (DHC-HGL) framework that captures heterogenous Context-Aware HAR (CA-HAR) hypergraph properties in a message-passing and neighborhood-aggregation fashion. Prior work only explored homogeneous or shallow-node-heterogeneous graphs. DHC-HGL handles heterogeneous CA-HAR data by innovatively 1) Constructing three different types of sub-hypergraphs that are each passed through different custom HyperGraph Convolution (HGC) layers designed to handle edge-heterogeneity and 2) Adopting a contrastive loss function to ensure node-heterogeneity. In rigorous evaluation on two CA-HAR datasets, DHC-HGL significantly outperformed state-of-the-art baselines by 5.8% to 16.7% on Matthews Correlation Coefficient (MCC) and 3.0% to 8.4% on Macro F1 scores. UMAP visualizations of learned CA-HAR node embeddings are also presented to enhance model explainability. Our code is publicly available1 to encourage further research.<\/jats:p>","DOI":"10.1145\/3631444","type":"journal-article","created":{"date-parts":[[2024,1,12]],"date-time":"2024-01-12T12:52:04Z","timestamp":1705063924000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Deep Heterogeneous Contrastive Hyper-Graph Learning for In-the-Wild Context-Aware Human Activity Recognition"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9247-1162","authenticated-orcid":false,"given":"Wen","family":"Ge","sequence":"first","affiliation":[{"name":"Worcester Polytechnic Institute, Worcester, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9987-0342","authenticated-orcid":false,"given":"Guanyi","family":"Mou","sequence":"additional","affiliation":[{"name":"Worcester Polytechnic Institute, Worcester, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3361-4952","authenticated-orcid":false,"given":"Emmanuel O.","family":"Agu","sequence":"additional","affiliation":[{"name":"Worcester Polytechnic Institute, Worcester, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9004-1740","authenticated-orcid":false,"given":"Kyumin","family":"Lee","sequence":"additional","affiliation":[{"name":"Worcester Polytechnic Institute, Worcester, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,1,12]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Proc. NIPS 30","year":"2017","unstructured":"2017. Lightgbm: A highly efficient gradient boosting decision tree. Proc. NIPS 30 (2017)."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSC.2020.00015"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3397323"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2020.107637"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2019.2920283"},{"key":"e_1_2_1_6_1","volume-title":"Int'l Conf. Automatic Face & Gesture Rec","author":"Chakraborty Bhaskar","unstructured":"Bhaskar Chakraborty, Ognjen Rudovic, and Jordi Gonzalez. 2008. View-invariant human-body detection with extension to human action recognition using component-wise HMM of body parts. In Int'l Conf. Automatic Face & Gesture Rec. IEEE, 1--6."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3380985"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1080\/10739149.2017.1363054"},{"key":"e_1_2_1_9_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3381012","article-title":"METIER: A deep multi-task learning based activity and user recognition model using wearable sensors","volume":"4","author":"Chen Ling","year":"2020","unstructured":"Ling Chen, Yi Zhang, and Liangying Peng. 2020. METIER: A deep multi-task learning based activity and user recognition model using wearable sensors. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 4, 1 (2020), 1--18.","journal-title":"Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"},{"key":"e_1_2_1_10_1","volume-title":"The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC genomics 21","author":"Chicco Davide","year":"2020","unstructured":"Davide Chicco and Giuseppe Jurman. 2020. The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC genomics 21 (2020), 1--13."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.smhl.2020.100118"},{"key":"e_1_2_1_12_1","first-page":"4125","article-title":"Heterogeneous hypergraph variational autoencoder for link prediction","volume":"44","author":"Fan Haoyi","year":"2021","unstructured":"Haoyi Fan, Fengbin Zhang, Yuxuan Wei, Zuoyong Li, Changqing Zou, Yue Gao, and Qionghai Dai. 2021. Heterogeneous hypergraph variational autoencoder for link prediction. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 8 (2021), 4125--4138.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.3390\/s19040947"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICMLA51294.2020.00122"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSC52841.2022.00014"},{"key":"e_1_2_1_16_1","doi-asserted-by":"crossref","unstructured":"Wen Ge Guanyi Mou Emmanuel O Agu and Kyumin Lee. 2023. Heterogeneous Hyper-Graph Neural Networks for Context-aware Human Activity Recognition. In PerCom.","DOI":"10.1109\/PerComWorkshops56833.2023.10150328"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2006.100"},{"key":"e_1_2_1_18_1","volume-title":"Proc. NIPS 30","author":"Hamilton Will","year":"2017","unstructured":"Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. Proc. NIPS 30 (2017)."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3550285"},{"key":"e_1_2_1_20_1","volume-title":"Supervised contrastive learning. Advances in neural information processing systems 33","author":"Khosla Prannay","year":"2020","unstructured":"Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020. Supervised contrastive learning. Advances in neural information processing systems 33 (2020), 18661--18673."},{"key":"e_1_2_1_21_1","volume-title":"Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907","author":"Kipf Thomas N","year":"2016","unstructured":"Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11604"},{"key":"e_1_2_1_23_1","volume-title":"DisenHCN: Disentangled Hypergraph Convolutional Networks for Spatiotemporal Activity Prediction. arXiv preprint arXiv:2208.06794","author":"Li Yinfeng","year":"2022","unstructured":"Yinfeng Li, Chen Gao, Quanming Yao, Tong Li, Depeng Jin, and Yong Li. 2022. DisenHCN: Disentangled Hypergraph Convolutional Networks for Spatiotemporal Activity Prediction. arXiv preprint arXiv:2208.06794 (2022)."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2184489.2184504"},{"key":"e_1_2_1_25_1","volume-title":"On the variance of the adaptive learning rate and beyond. arXiv preprint arXiv:1908.03265","author":"Liu Liyuan","year":"2019","unstructured":"Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Jiawei Han. 2019. On the variance of the adaptive learning rate and beyond. arXiv preprint arXiv:1908.03265 (2019)."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3631429"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2019.12.150"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3534678.3539299"},{"key":"e_1_2_1_29_1","volume-title":"Spatiotemporal workshop co-located with NIPS.","author":"Martin Henry","year":"2018","unstructured":"Henry Martin, Dominik Bucher, Esra Suel, Pengxiang Zhao, Fernando Perez-Cruz, and Martin Raubal. 2018. Graph convolutional neural networks for human activity purpose imputation. In Spatiotemporal workshop co-located with NIPS."},{"key":"e_1_2_1_30_1","volume-title":"Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426","author":"McInnes Leland","year":"2018","unstructured":"Leland McInnes, John Healy, and James Melville. 2018. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03426 (2018)."},{"key":"e_1_2_1_31_1","volume-title":"Proc. PerCom. IEEE, 335--40","author":"Mohamed Abduallah","year":"2022","unstructured":"Abduallah Mohamed, Fernando Lejarza, Stephanie Cahail, Christian Claudel, and Edison Thomaz. 2022. HAR-GCNN: Deep Graph CNNs for Human Activity Recog. From Highly Unlabeled Mobile Sensor Data. In Proc. PerCom. IEEE, 335--40."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2494091.2494120"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.4249\/scholarpedia.1883"},{"key":"e_1_2_1_34_1","volume-title":"PerCom","author":"Ramamurthy Sreenivasan Ramasamy","unstructured":"Sreenivasan Ramasamy Ramamurthy, Soumyajit Chatterjee, Elizabeth Galik, Aryya Gangopadhyay, Nirmalya Roy, Bivas Mitra, and Sandip Chakraborty. 2022. CogAx: Early Assessment of Cognitive and Functional Impairment from Accelerometry. In PerCom. IEEE, 66--76."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.pmcj.2016.08.003"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2018.2833443"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2020.3028207"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1038\/s42005-021-00788-w"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3437963.3441835"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/MPRV.2017.3971131"},{"key":"e_1_2_1_41_1","first-page":"1","article-title":"Context recognition in-the-wild: Unified model for multi-modal sensors and multi-label classification","volume":"1","author":"Vaizman Yonatan","year":"2018","unstructured":"Yonatan Vaizman, Nadir Weibel, and Gert Lanckriet. 2018. Context recognition in-the-wild: Unified model for multi-modal sensors and multi-label classification. ACM IMWUT 1, 4 (2018), 1--22.","journal-title":"ACM IMWUT"},{"key":"e_1_2_1_42_1","volume-title":"Graph attention networks. arXiv preprint arXiv:1710.10903","author":"Veli\u010dkovi\u0107 Petar","year":"2017","unstructured":"Petar Veli\u010dkovi\u0107, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. 2017. Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01020"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313635"},{"key":"e_1_2_1_45_1","volume-title":"Heterogeneous hypergraph embedding for location-based social networks. Trans. Knowledge and Data Engineering","author":"Yang Dingqi","year":"2020","unstructured":"Dingqi Yang, Bingqing Qu, Jie Yang, and Philippe Cudr\u00e9-Mauroux. 2020. Lbsn2vec++: Heterogeneous hypergraph embedding for location-based social networks. Trans. Knowledge and Data Engineering (2020)."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3161181"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330961"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3569482"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2021.10.009"},{"key":"e_1_2_1_50_1","volume-title":"Graph neural networks: A review of methods and applications. AI open 1","author":"Zhou Jie","year":"2020","unstructured":"Jie Zhou, Ganqu Cui, Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. 2020. Graph neural networks: A review of methods and applications. AI open 1 (2020), 57--81."},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2016.07.030"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIFS.2020.2985628"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3631444","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3631444","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,27]],"date-time":"2025-08-27T16:58:31Z","timestamp":1756313911000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3631444"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,19]]},"references-count":52,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,12,19]]}},"alternative-id":["10.1145\/3631444"],"URL":"https:\/\/doi.org\/10.1145\/3631444","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,19]]},"assertion":[{"value":"2024-01-12","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}