{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,11]],"date-time":"2026-03-11T17:47:21Z","timestamp":1773251241711,"version":"3.50.1"},"reference-count":58,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2022,9,27]],"date-time":"2022-09-27T00:00:00Z","timestamp":1664236800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100013297","name":"FCT\u2014Funda\u00e7\u00e3o para a Ci\u00eancia e a Tecnologia","doi-asserted-by":"publisher","award":["E!114310"],"award-info":[{"award-number":["E!114310"]}],"id":[{"id":"10.13039\/100013297","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Human Activity Recognition (HAR) has been studied extensively, yet current approaches are not capable of generalizing across different domains (i.e., subjects, devices, or datasets) with acceptable performance. This lack of generalization hinders the applicability of these models in real-world environments. As deep neural networks are becoming increasingly popular in recent work, there is a need for an explicit comparison between handcrafted and deep representations in Out-of-Distribution (OOD) settings. This paper compares both approaches in multiple domains using homogenized public datasets. First, we compare several metrics to validate three different OOD settings. In our main experiments, we then verify that even though deep learning initially outperforms models with handcrafted features, the situation is reversed as the distance from the training distribution increases. These findings support the hypothesis that handcrafted features may generalize better across specific domains.<\/jats:p>","DOI":"10.3390\/s22197324","type":"journal-article","created":{"date-parts":[[2022,9,28]],"date-time":"2022-09-28T03:30:37Z","timestamp":1664335837000},"page":"7324","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":28,"title":["Comparing Handcrafted Features and Deep Neural Representations for Domain Generalization in Human Activity Recognition"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7279-1890","authenticated-orcid":false,"given":"Nuno","family":"Bento","sequence":"first","affiliation":[{"name":"Associa\u00e7\u00e3o Fraunhofer Portugal Research, Rua Alfredo Allen 455\/461, 4200-135 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0385-053X","authenticated-orcid":false,"given":"Joana","family":"Rebelo","sequence":"additional","affiliation":[{"name":"Associa\u00e7\u00e3o Fraunhofer Portugal Research, Rua Alfredo Allen 455\/461, 4200-135 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9445-4809","authenticated-orcid":false,"given":"Mar\u00edlia","family":"Barandas","sequence":"additional","affiliation":[{"name":"Associa\u00e7\u00e3o Fraunhofer Portugal Research, Rua Alfredo Allen 455\/461, 4200-135 Porto, Portugal"},{"name":"Laborat\u00f3rio de Instrumenta\u00e7\u00e3o, Engenharia Biom\u00e9dica e F\u00edsica da Radia\u00e7\u00e3o (LIBPhys\u2013UNL), Departamento de F\u00edsica, Faculdade de Ci\u00eancias e Tecnologia (FCT), Universidade Nova de Lisboa, 2829-516 Caparica, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4234-5336","authenticated-orcid":false,"given":"Andr\u00e9 V.","family":"Carreiro","sequence":"additional","affiliation":[{"name":"Associa\u00e7\u00e3o Fraunhofer Portugal Research, Rua Alfredo Allen 455\/461, 4200-135 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0027-5157","authenticated-orcid":false,"given":"Andrea","family":"Campagner","sequence":"additional","affiliation":[{"name":"Dipartimento di Informatica, Sistemistica e Comunicazione, Universit\u00e0 degli Studi di Milano-Bicocca, 20126 Milan, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4065-3415","authenticated-orcid":false,"given":"Federico","family":"Cabitza","sequence":"additional","affiliation":[{"name":"Dipartimento di Informatica, Sistemistica e Comunicazione, Universit\u00e0 degli Studi di Milano-Bicocca, 20126 Milan, Italy"},{"name":"IRCCS Istituto Ortopedico Galeazzi, 20161 Milan, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4022-7424","authenticated-orcid":false,"given":"Hugo","family":"Gamboa","sequence":"additional","affiliation":[{"name":"Associa\u00e7\u00e3o Fraunhofer Portugal Research, Rua Alfredo Allen 455\/461, 4200-135 Porto, Portugal"},{"name":"Laborat\u00f3rio de Instrumenta\u00e7\u00e3o, Engenharia Biom\u00e9dica e F\u00edsica da Radia\u00e7\u00e3o (LIBPhys\u2013UNL), Departamento de F\u00edsica, Faculdade de Ci\u00eancias e Tecnologia (FCT), Universidade Nova de Lisboa, 2829-516 Caparica, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2022,9,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Sousa Lima, W., Souto, E., El-Khatib, K., Jalali, R., and Gama, J. (2019). Human activity recognition using inertial sensors in a smartphone: An overview. Sensors, 19.","DOI":"10.3390\/s19143213"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Ariza-Colpas, P.P., Vicario, E., Oviedo-Carrascal, A.I., Butt Aziz, S., Pi\u00f1eres-Melo, M.A., Quintero-Linero, A., and Patara, F. (2022). Human Activity Recognition Data Analysis: History, Evolutions, and New Trends. Sensors, 22.","DOI":"10.3390\/s22093401"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"256","DOI":"10.12720\/ijsps.1.2.256-262","article-title":"Reviews on various inertial measurement unit (IMU) sensor applications","volume":"1","author":"Ahmad","year":"2013","journal-title":"Int. J. Signal Process. Syst."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Li, F., Shirahama, K., Nisar, M.A., K\u00f6ping, L., and Grzegorzek, M. (2018). Comparison of feature learning methods for human activity recognition using wearable sensors. Sensors, 18.","DOI":"10.3390\/s18020679"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1016\/j.neucom.2020.10.056","article-title":"Cross-subject transfer learning in human activity recognition systems using generative adversarial networks","volume":"426","author":"Soleimani","year":"2021","journal-title":"Neurocomputing"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Wang, J., Zheng, V.W., Chen, Y., and Huang, M. (2018, January 28\u201331). Deep transfer learning for cross-domain activity recognition. Proceedings of the 3rd International Conference on Crowd Science and Engineering, Singapore.","DOI":"10.1145\/3265689.3265705"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Hoelzemann, A., and Van Laerhoven, K. (2020, January 12\u201316). Digging deeper: Towards a better understanding of transfer learning for human activity recognition. Proceedings of the 2020 International Symposium on Wearable Computers, Virtual.","DOI":"10.1145\/3410531.3414311"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.patrec.2018.02.010","article-title":"Deep learning for sensor-based activity recognition: A survey","volume":"119","author":"Wang","year":"2019","journal-title":"Pattern Recognit. Lett."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Nafea, O., Abdul, W., Muhammad, G., and Alsulaiman, M. (2021). Sensor-based human activity recognition with spatio-temporal deep learning. Sensors, 21.","DOI":"10.3390\/s21062141"},{"key":"ref_10","unstructured":"Sagawa, S., Raghunathan, A., Koh, P.W., and Liang, P. (2020, January 13\u201318). An investigation of why overparameterization exacerbates spurious correlations. Proceedings of the 37th International Conference on Machine Learning, Virtual Conference."},{"key":"ref_11","unstructured":"Arjovsky, M., Bottou, L., Gulrajani, I., and Lopez-Paz, D. (2019). Invariant risk minimization. arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"100456","DOI":"10.1016\/j.softx.2020.100456","article-title":"TSFEL: Time series feature extraction library","volume":"11","author":"Barandas","year":"2020","journal-title":"SoftwareX"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Chen, Y., and Xue, Y. (2015, January 9\u201312). A deep learning approach to human activity recognition based on single accelerometer. Proceedings of the 2015 IEEE International Conference on Systems, Man, and Cybernetics, Hong Kong, China.","DOI":"10.1109\/SMC.2015.263"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zebin, T., Scully, P.J., and Ozanyan, K.B. (2016\u20133, January 30). Human activity recognition with inertial sensors using a deep learning approach. Proceedings of the 2016 IEEE Sensors, Orlando, FL, USA.","DOI":"10.1109\/ICSENS.2016.7808590"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"665","DOI":"10.1038\/s42256-020-00257-z","article-title":"Shortcut learning in deep neural networks","volume":"2","author":"Geirhos","year":"2020","journal-title":"Nat. Mach. Intell."},{"key":"ref_16","unstructured":"Gagnon-Audet, J.C., Ahuja, K., Darvishi-Bayazi, M.J., Dumas, G., and Rish, I. (2022). WOODS: Benchmarks for Out-of-Distribution Generalization in Time Series Tasks. arXiv."},{"key":"ref_17","unstructured":"Lee, S.M., Yoon, S.M., and Cho, H. (2017, January 13\u201316). Human activity recognition from accelerometer data using Convolutional Neural Network. Proceedings of the 2017 IEEE International Conference on Big Data and Smart Computing (Bigcomp), Jeju, Korea."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Ferrari, A., Micucci, D., Mobilio, M., and Napoletano, P. (2019, January 19\u201321). Hand-crafted features vs residual networks for human activities recognition using accelerometer. Proceedings of the 2019 IEEE 23rd International Symposium on Consumer Technologies (ISCT), Ancona, Italy.","DOI":"10.1109\/ISCE.2019.8901021"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1007\/s10994-008-5059-5","article-title":"A theory of learning with similarity functions","volume":"72","author":"Balcan","year":"2008","journal-title":"Mach. Learn."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1011","DOI":"10.1080\/02664760802192981","article-title":"Diagnostics of prior-data agreement in applied Bayesian analysis","volume":"35","author":"Bousquet","year":"2008","journal-title":"J. Appl. Stat."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Kouw, W.M., Loog, M., Bartels, L.W., and Mendrik, A.M. (2019, January 8\u201311). Learning an MR acquisition-invariant representation using Siamese neural networks. Proceedings of the 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019), Venice, Italy.","DOI":"10.1109\/ISBI.2019.8759281"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Redko, I., Morvant, E., Habrard, A., Sebban, M., and Bennani, Y. (2019). Advances in Domain Adaptation Theory, Elsevier.","DOI":"10.1016\/B978-1-78548-236-6.50002-7"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Veen, D., Stoel, D., Schalken, N., Mulder, K., and Van de Schoot, R. (2018). Using the data agreement criterion to rank experts\u2019 beliefs. Entropy, 20.","DOI":"10.3390\/e20080592"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Schat, E., van de Schoot, R., Kouw, W.M., Veen, D., and Mendrik, A.M. (2020). The data representativeness criterion: Predicting the performance of supervised classification based on data set similarity. PLoS ONE, 15.","DOI":"10.1371\/journal.pone.0237009"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"106288","DOI":"10.1016\/j.cmpb.2021.106288","article-title":"The importance of being external. methodological insights for the external validation of machine learning models in medicine","volume":"208","author":"Cabitza","year":"2021","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_26","unstructured":"Tzeng, E., Hoffman, J., Zhang, N., Saenko, K., and Darrell, T. (2014). Deep domain confusion: Maximizing for domain invariance. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Zhou, F., Jiang, Z., Shui, C., Wang, B., and Chaib-draa, B. (2020). Domain generalization with optimal transport and metric learning. arXiv.","DOI":"10.1016\/j.neucom.2020.09.091"},{"key":"ref_28","unstructured":"Shen, Z., Liu, J., He, Y., Zhang, X., Xu, R., Yu, H., and Cui, P. (2021). Towards out-of-distribution generalization: A survey. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1016\/j.eswa.2016.04.032","article-title":"Human activity recognition with smartphone sensors using deep learning neural networks","volume":"59","author":"Ronao","year":"2016","journal-title":"Expert Syst. Appl."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Logacjov, A., Bach, K., Kongsvold, A., B\u00e5rdstu, H.B., and Mork, P.J. (2021). HARTH: A Human Activity Recognition Dataset for Machine Learning. Sensors, 21.","DOI":"10.3390\/s21237853"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"9893","DOI":"10.1109\/ACCESS.2018.2890675","article-title":"InnoHAR: A deep neural network for complex human activity recognition","volume":"7","author":"Xu","year":"2019","journal-title":"IEEE Access"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Moreira, D., Barandas, M., Rocha, T., Alves, P., Santos, R., Leonardo, R., Vieira, P., and Gamboa, H. (2021). Human Activity Recognition for Indoor Localization Using Smartphone Inertial Sensors. Sensors, 21.","DOI":"10.3390\/s21186316"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"133982","DOI":"10.1109\/ACCESS.2020.3010715","article-title":"Deep neural networks for human activity recognition with wearable sensors: Leave-one-subject-out cross-validation for model selection","volume":"8","author":"Gholamiangonabadi","year":"2020","journal-title":"IEEE Access"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Bragan\u00e7a, H., Colonna, J.G., Oliveira, H.A.B.F., and Souto, E. (2022). How Validation Methodology Influences Human Activity Recognition Mobile Systems. Sensors, 22.","DOI":"10.3390\/s22062360"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Ding, R., Li, X., Nie, L., Li, J., Si, X., Chu, D., Liu, G., and Zhan, D. (2018). Empirical study and improvement on deep transfer learning for human activity recognition. Sensors, 19.","DOI":"10.3390\/s19010057"},{"key":"ref_36","first-page":"3438","article-title":"Invariance principle meets information bottleneck for out-of-distribution generalization","volume":"34","author":"Ahuja","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_37","unstructured":"Rosenfeld, E., Ravikumar, P., and Risteski, A. (2020). The risks of invariant risk minimization. arXiv."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Boyer, P., Burns, D., and Whyne, C. (2021). Out-of-distribution detection of human activity recognition with smartwatch inertial sensors. Sensors, 21.","DOI":"10.3390\/s21051669"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Trabelsi, I., Fran\u00e7oise, J., and Bellik, Y. (2022, January 22\u201324). Sensor-based Activity Recognition using Deep Learning: A Comparative Study. Proceedings of the 8th International Conference on Movement and Computing, Chicago, IL, USA.","DOI":"10.1145\/3537972.3537996"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Reiss, A., and Stricker, D. (2012, January 18\u201322). Introducing a New Benchmarked Dataset for Activity Monitoring. Proceedings of the 2012 16th Annual International Symposium on Wearable Computers (ISWC), Newcastle, UK.","DOI":"10.1109\/ISWC.2012.13"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Reiss, A., and Stricker, D. (2012, January 6\u20138). Creating and Benchmarking a New Dataset for Physical Activity Monitoring. Proceedings of the 5th International Conference on PErvasive Technologies Related to Assistive Environments, Heraklion, Greece.","DOI":"10.1145\/2413097.2413148"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"10146","DOI":"10.3390\/s140610146","article-title":"Fusion of smartphone motion sensors for physical activity recognition","volume":"14","author":"Shoaib","year":"2014","journal-title":"Sensors"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Leutheuser, H., Schuldhaus, D., and Eskofier, B.M. (2013). Hierarchical, Multi-Sensor Based Classification of Daily Life Activities: Comparison with State-of-the-Art Algorithms Using a Benchmark Dataset. PLoS ONE, 8.","DOI":"10.1371\/journal.pone.0075196"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Pecchia, L., Chen, L.L., Nugent, C., and Bravo, J. (2014). mHealthDroid: A Novel Framework for Agile Development of Mobile Health Applications. Ambient Assisted Living and Daily Activities, Springer International Publishing.","DOI":"10.1007\/978-3-319-13105-4"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Banos, O., Villalonga, C., Garcia, R., Saez, A., Damas, M., Holgado-Terriza, J.A., Lee, S., Pomares, H., and Rojas, I. (2015). Design, implementation and validation of a novel open framework for agile development of mobile health applications. BioMed. Eng. Online, 14.","DOI":"10.1186\/1475-925X-14-S2-S6"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Sztyler, T., and Stuckenschmidt, H. (2016, January 14\u201319). On-body Localization of Wearable Devices: An Investigation of Position-Aware Activity Recognition. Proceedings of the 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom), Sydney, NSW, Australia.","DOI":"10.1109\/PERCOM.2016.7456521"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Ferrari, A., Mobilio, M., Micucci, D., and Napoletano, P. (2019, January 8\u201313). On the homogenization of heterogeneous inertial-based databases for human activity recognition. Proceedings of the 2019 IEEE World Congress on Services (SERVICES), Milan, Italy.","DOI":"10.1109\/SERVICES.2019.00084"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Figueira, C., Matias, R., and Gamboa, H. (2016, January 21\u201323). Body Location Independent Activity Monitoring. Proceedings of the 9th International Conference on Bio-inspired Systems and Signal Processing, Rome, Italy.","DOI":"10.5220\/0005699601900197"},{"key":"ref_49","first-page":"577","article-title":"Comparative study of machine learning and deep learning architecture for human activity recognition using accelerometer data","volume":"8","author":"Shakya","year":"2018","journal-title":"Int. J. Mach. Learn. Comput."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Shiranthika, C., Premakumara, N., Chiu, H.L., Samani, H., Shyalika, C., and Yang, C.Y. (2020, January 2\u20134). Human Activity Recognition Using CNN & LSTM. Proceedings of the 2020 5th International Conference on Information Technology Research (ICITR), Online.","DOI":"10.1109\/ICITR51448.2020.9310792"},{"key":"ref_51","unstructured":"Arjovsky, M., Chintala, S., and Bottou, L. (2017, January 6\u201311). Wasserstein generative adversarial networks. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_52","first-page":"723","article-title":"A kernel two-sample test","volume":"13","author":"Gretton","year":"2012","journal-title":"J. Mach. Learn. Res."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s12911-020-01224-9","article-title":"As if sand were stone. New concepts and metrics to probe the ground on which to build trustable AI","volume":"20","author":"Cabitza","year":"2020","journal-title":"BMC Med. Inform. Decis. Mak."},{"key":"ref_54","unstructured":"Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019, January 8\u201314). Pytorch: An imperative style, high-performance deep learning library. Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, BC, Canada."},{"key":"ref_55","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_56","unstructured":"Heckel, R., and Yilmaz, F.F. (2020). Early stopping in deep networks: Double descent and how to eliminate it. arXiv."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"124003","DOI":"10.1088\/1742-5468\/ac3a74","article-title":"Deep double descent: Where bigger models and more data hurt","volume":"2021","author":"Nakkiran","year":"2021","journal-title":"J. Stat. Mech. Theory Exp."},{"key":"ref_58","unstructured":"Bai, J., Lu, F., and Zhang, K. (2022, September 26). ONNX: Open Neural Network Exchange. Available online: https:\/\/github.com\/onnx\/onnx."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/19\/7324\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:40:22Z","timestamp":1760143222000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/19\/7324"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,27]]},"references-count":58,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2022,10]]}},"alternative-id":["s22197324"],"URL":"https:\/\/doi.org\/10.3390\/s22197324","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,27]]}}}