{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T15:27:05Z","timestamp":1774538825027,"version":"3.50.1"},"reference-count":63,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2023,12,19]],"date-time":"2023-12-19T00:00:00Z","timestamp":1702944000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."],"published-print":{"date-parts":[[2023,12,19]]},"abstract":"<jats:p>Deep Learning models are a standard solution for sensor-based Human Activity Recognition (HAR), but their deployment is often limited by labeled data scarcity and models' opacity. Neuro-Symbolic AI (NeSy) provides an interesting research direction to mitigate these issues by infusing knowledge about context information into HAR deep learning classifiers. However, existing NeSy methods for context-aware HAR require computationally expensive symbolic reasoners during classification, making them less suitable for deployment on resource-constrained devices (e.g., mobile devices). Additionally, NeSy approaches for context-aware HAR have never been evaluated on in-the-wild datasets, and their generalization capabilities in real-world scenarios are questionable. In this work, we propose a novel approach based on a semantic loss function that infuses knowledge constraints in the HAR model during the training phase, avoiding symbolic reasoning during classification. Our results on scripted and in-the-wild datasets show the impact of different semantic loss functions in outperforming a purely data-driven model. We also compare our solution with existing NeSy methods and analyze each approach's strengths and weaknesses. Our semantic loss remains the only NeSy solution that can be deployed as a single DNN without the need for symbolic reasoning modules, reaching recognition rates close (and better in some cases) to existing approaches.<\/jats:p>","DOI":"10.1145\/3631407","type":"journal-article","created":{"date-parts":[[2024,1,12]],"date-time":"2024-01-12T12:52:04Z","timestamp":1705063924000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["Semantic Loss"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5207-566X","authenticated-orcid":false,"given":"Luca","family":"Arrotta","sequence":"first","affiliation":[{"name":"University of Milan, Via Celoria, Milan, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8247-2524","authenticated-orcid":false,"given":"Gabriele","family":"Civitarese","sequence":"additional","affiliation":[{"name":"University of Milan, Via Celoria, Milan, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1727-7650","authenticated-orcid":false,"given":"Claudio","family":"Bettini","sequence":"additional","affiliation":[{"name":"University of Milan, Via Celoria, Milan, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,1,12]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3158645"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-19-0361-8_3"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2020.03.289"},{"key":"e_1_2_1_4_1","first-page":"29944","article-title":"Semantic probabilistic layers for neuro-symbolic learning","volume":"35","author":"Ahmed Kareem","year":"2022","unstructured":"Kareem Ahmed, Stefano Teso, Kai-Wei Chang, Guy Van den Broeck, and Antonio Vergari. 2022. Semantic probabilistic layers for neuro-symbolic learning. Advances in Neural Information Processing Systems 35 (2022), 29944--29959.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_5_1","volume-title":"Knowledge Infusion for Context-Aware Sensor-Based Human Activity Recognition. In 2022 IEEE International Conference on Smart Computing (SMARTCOMP). IEEE, 1--8.","author":"Arrotta Luca","year":"2022","unstructured":"Luca Arrotta, Gabriele Civitarese, and Claudio Bettini. 2022. Knowledge Infusion for Context-Aware Sensor-Based Human Activity Recognition. In 2022 IEEE International Conference on Smart Computing (SMARTCOMP). IEEE, 1--8."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.pmcj.2023.101780"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/MDM58254.2023.00063"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSEN.2020.2964278"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/FIOT.2018.8325603"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2861004"},{"key":"e_1_2_1_11_1","volume-title":"A survey of context modelling and reasoning techniques. Pervasive and mobile computing 6, 2","author":"Bettini Claudio","year":"2010","unstructured":"Claudio Bettini, Oliver Brdiczka, Karen Henricksen, Jadwiga Indulska, Daniela Nicklas, Anand Ranganathan, and Daniele Riboni. 2010. A survey of context modelling and reasoning techniques. Pervasive and mobile computing 6, 2 (2010), 161--180."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3015091"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2020.105816"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.pmcj.2014.09.010"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.websem.2015.09.002"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jpdc.2017.05.007"},{"key":"e_1_2_1_17_1","volume-title":"This looks like that: deep learning for interpretable image recognition. Advances in neural information processing systems 32","author":"Chen Chaofan","year":"2019","unstructured":"Chaofan Chen, Oscar Li, Daniel Tao, Alina Barnett, Cynthia Rudin, and Jonathan K Su. 2019. This looks like that: deep learning for interpretable image recognition. Advances in neural information processing systems 32 (2019)."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447744"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2019.2930050"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/s42486-020-00026-2"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-021-04590-0"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-021-05966-z"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2021.09.022"},{"key":"e_1_2_1_24_1","volume-title":"International Conference on Machine Learning. PMLR","author":"Fischer Marc","year":"2019","unstructured":"Marc Fischer, Mislav Balunovic, Dana Drachsler-Cohen, Timon Gehr, Ce Zhang, and Martin Vechev. 2019. DL2: training and querying neural networks with logic. In International Conference on Machine Learning. PMLR, 1931--1941."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2020.3031769"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2017.01.025"},{"key":"e_1_2_1_27_1","volume-title":"Coherent hierarchical multi-label classification networks. Advances in neural information processing systems 33","author":"Giunchiglia Eleonora","year":"2020","unstructured":"Eleonora Giunchiglia and Thomas Lukasiewicz. 2020. Coherent hierarchical multi-label classification networks. Advances in neural information processing systems 33 (2020), 9662--9673."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/SMC.2015.525"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.3390\/s21217278"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3550299"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1007\/11575771_53"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3550294"},{"key":"e_1_2_1_33_1","first-page":"173","article-title":"Neuro-Symbolic AI= Neural+ Logical+ Probabilistic AI","volume":"342","author":"Hitzler P","year":"2022","unstructured":"P Hitzler and M Sarker. 2022. Neuro-Symbolic AI= Neural+ Logical+ Probabilistic AI. Neuro-Symbolic Artificial Intelligence: The State of the Art 342 (2022), 173.","journal-title":"Neuro-Symbolic Artificial Intelligence: The State of the Art"},{"key":"e_1_2_1_34_1","volume-title":"Harnessing deep neural networks with logic rules. arXiv preprint arXiv:1603.06318","author":"Hu Zhiting","year":"2016","unstructured":"Zhiting Hu, Xuezhe Ma, Zhengzhong Liu, Eduard Hovy, and Eric Xing. 2016. Harnessing deep neural networks with logic rules. arXiv preprint arXiv:1603.06318 (2016)."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3517246"},{"key":"e_1_2_1_36_1","volume-title":"Neuro-Symbolic Artificial Intelligence: Current Trends. arXiv e-prints","author":"Sarker Md Kamruzzaman","year":"2021","unstructured":"Md Kamruzzaman Sarker, Lu Zhou, Aaron Eberhart, and Pascal Hitzler. 2021. Neuro-Symbolic Artificial Intelligence: Current Trends. arXiv e-prints (2021), arXiv--2105."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-022-01756-8"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2021.3086590"},{"key":"e_1_2_1_39_1","volume-title":"Ian Horrocks, Zhe Wu, Achille Fokoue, Carsten Lutz, et al.","author":"Motik Boris","year":"2009","unstructured":"Boris Motik, Bernardo Cuenca Grau, Ian Horrocks, Zhe Wu, Achille Fokoue, Carsten Lutz, et al. 2009. OWL 2 web ontology language profiles. W3C recommendation 27, 61 (2009)."},{"key":"e_1_2_1_40_1","unstructured":"Mathias Niepert Jan Noessner and Heiner Stuckenschmidt. 2011. Log-linear description logics. In IJCAI. 2153--2158."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-23580-1_25"},{"key":"e_1_2_1_42_1","doi-asserted-by":"crossref","unstructured":"Nobuyuki Oishi Daniel Roggen Philip Birch and Paula Lago. 2023. Learning Using Privileged Information for Wearable-based Human Activity Recognition. In 2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops). IEEE 233--234.","DOI":"10.1109\/PerComWorkshops56833.2023.10150397"},{"key":"e_1_2_1_43_1","volume-title":"Rise: Randomized input sampling for explanation of black-box models. arXiv preprint arXiv:1806.07421","author":"Petsiuk Vitali","year":"2018","unstructured":"Vitali Petsiuk, Abir Das, and Kate Saenko. 2018. Rise: Randomized input sampling for explanation of black-box models. arXiv preprint arXiv:1806.07421 (2018)."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00779-010-0331-7"},{"key":"e_1_2_1_46_1","volume-title":"Human activity recognition with smartphone sensors using deep learning neural networks. Expert systems with applications 59","author":"Ronao Charissa Ann","year":"2016","unstructured":"Charissa Ann Ronao and Sung-Bae Cho. 2016. Human activity recognition with smartphone sensors using deep learning neural networks. Expert systems with applications 59 (2016), 235--244."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2490832"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2021.3053704"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.74"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIC.2019.2960071"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.websem.2007.03.004"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2020.10.056"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.3233\/JIFS-169976"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2020.113426"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/MPRV.2017.3971131"},{"key":"e_1_2_1_56_1","volume-title":"A new learning paradigm: Learning using privileged information. Neural networks 22, 5-6","author":"Vapnik Vladimir","year":"2009","unstructured":"Vladimir Vapnik and Akshay Vashist. 2009. A new learning paradigm: Learning using privileged information. Neural networks 22, 5-6 (2009), 544--557."},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2018.02.010"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01081-2_30"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cogsys.2018.04.003"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3384419.3431158"},{"key":"e_1_2_1_61_1","volume-title":"International conference on machine learning. PMLR, 5502--5511","author":"Xu Jingyi","year":"2018","unstructured":"Jingyi Xu, Zilu Zhang, Tal Friedman, Yitao Liang, and Guy Broeck. 2018. A semantic loss function for deep learning with symbolic knowledge. In International conference on machine learning. PMLR, 5502--5511."},{"key":"e_1_2_1_62_1","volume-title":"Phyo Phyo San, Xiao Li Li, and Shonali Krishnaswamy.","author":"Yang Jianbo","year":"2015","unstructured":"Jianbo Yang, Minh Nhut Nguyen, Phyo Phyo San, Xiao Li Li, and Shonali Krishnaswamy. 2015. Deep convolutional neural networks on multichannel time series for human activity recognition. In Twenty-fourth international joint conference on artificial intelligence."},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.21629\/JSEE.2017.01.18"}],"container-title":["Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3631407","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3631407","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,27]],"date-time":"2025-08-27T16:58:17Z","timestamp":1756313897000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3631407"}},"subtitle":["A New Neuro-Symbolic Approach for Context-Aware Human Activity Recognition"],"short-title":[],"issued":{"date-parts":[[2023,12,19]]},"references-count":63,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,12,19]]}},"alternative-id":["10.1145\/3631407"],"URL":"https:\/\/doi.org\/10.1145\/3631407","relation":{},"ISSN":["2474-9567"],"issn-type":[{"value":"2474-9567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,19]]},"assertion":[{"value":"2024-01-12","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}