{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T14:39:13Z","timestamp":1777559953630,"version":"3.51.4"},"reference-count":50,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2024,12,5]],"date-time":"2024-12-05T00:00:00Z","timestamp":1733356800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>With the increasing availability of wearable devices for data collection, studies in human activity recognition have gained significant popularity. These studies report high accuracies on k-fold cross validation, which is not reflective of their generalization performance but is a result of the inappropriate split of testing and training datasets, causing these models to evaluate the same subjects that they were trained on, making them subject-dependent. This study comparatively discusses this validation approach with a universal approach, Leave-One-Subject-Out (LOSO) cross-validation which is not subject-dependent and ensures that an entirely new subject is used for evaluation in each fold, validated on four different machine learning models trained on windowed data and select hand-crafted features. The random forest model, with the highest accuracy of 76% when evaluated on LOSO, achieved an accuracy of 89% on k-fold cross-validation, demonstrating data leakage. Additionally, this experiment underscores the significance of hand-crafted features by contrasting their accuracy with that of raw sensor models. The feature models demonstrate a remarkable 30% higher accuracy, underscoring the importance of feature engineering in enhancing the robustness and precision of HAR systems.<\/jats:p>","DOI":"10.3390\/a17120556","type":"journal-article","created":{"date-parts":[[2024,12,5]],"date-time":"2024-12-05T06:24:44Z","timestamp":1733379884000},"page":"556","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors"],"prefix":"10.3390","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-4566-7144","authenticated-orcid":false,"given":"Saeed Ur","family":"Rehman","sequence":"first","affiliation":[{"name":"Faculty of Science and Engineering, University of Hull, Cottingham Rd., Hull HU6 7RX, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anwar","family":"Ali","sequence":"additional","affiliation":[{"name":"Department of Electronic and Electrical Engineering, Swansea University Bay Campus, Swansea SA1 8EN, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Adil Mehmood","family":"Khan","sequence":"additional","affiliation":[{"name":"Faculty of Science and Engineering, University of Hull, Cottingham Rd., Hull HU6 7RX, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Cynthia","family":"Okpala","sequence":"additional","affiliation":[{"name":"Faculty of Science and Engineering, University of Hull, Cottingham Rd., Hull HU6 7RX, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,12,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1109\/MPRV.2002.993141","article-title":"The computer for the 21st Century","volume":"1","author":"Weiser","year":"2002","journal-title":"IEEE Pervasive Comput."},{"key":"ref_2","unstructured":"Jiang, S., Shull, P.B., Lv, B., Sheng, X., Zhang, C., and Wang, H. (2016, January 3\u20137). Development of a real-time hand gesture recognition wristband based on sEMG and IMU sensing. Proceedings of the 16 IEEE International Conference on Robotics and Biomimetics (ROBIO), Qingdao, China. Available online: https:\/\/ieeexplore.ieee.org\/document\/7866498."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2499621","article-title":"A tutorial on human activity recognition using body-worn inertial sensors","volume":"46","author":"Bulling","year":"2014","journal-title":"ACM Comput. Surv."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Bragan\u00e7a, H., Colonna, J.G., Oliveira, H.A.B.F., and Souto, E. (2022). How Validation Methodology Influences Human Activity Recognition Mobile Systems. Sensors, 22.","DOI":"10.3390\/s22062360"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Wang, D., Candinegara, E., Hou, J., Tan, A.-H., and Miao, C. (2017, January 15\u201317). Robust Human Activity Recognition Using Lesser Number of Wearable Sensors. Proceedings of the 2017 International Conference on Security, Pattern Analysis, and Cybernetics (SPAC), Shenzhen, China. Available online: https:\/\/ink.library.smu.edu.sg\/sis_research\/5468\/.","DOI":"10.1109\/SPAC.2017.8304292"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1419","DOI":"10.1109\/TKDE.2019.2904256","article-title":"Multi-view Scaling Support Vector Machines for Classification and Feature Selection","volume":"32","author":"Xu","year":"2019","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1214\/09-SS054","article-title":"A survey of cross-validation procedures for model selection","volume":"4","author":"Arlot","year":"2010","journal-title":"Stat. Surv."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1007\/s40860-021-00147-0","article-title":"Trends in human activity recognition using smartphones","volume":"7","author":"Ferrari","year":"2021","journal-title":"J. Reliab. Intell. Environ."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"gix020","DOI":"10.1093\/gigascience\/gix020","article-title":"Using and understanding cross-validation strategies. Perspectives on Saeb et al","volume":"6","author":"Little","year":"2017","journal-title":"GigaScience"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Hammerla, N.Y., and Pl\u00f6tz, T. (2015, January 7\u201311). Let\u2019s (not) stick together: Pairwise similarity biases cross-validation in activity recognition. Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Osaka, Japan.","DOI":"10.1145\/2750858.2807551"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Widhalm, P., Leodolter, M., and Br\u00e4ndle, N. (2018, January 8\u201312). Top in the Lab, Flop in the Field? Evaluation of a sensor-based travel activity classifier with the SHL dataset. Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers, Singapore.","DOI":"10.1145\/3267305.3267514"},{"key":"ref_12","unstructured":"Dehghani, A., Glatard, T., and Shihab, E. (2019). Subject Cross Validation in Human Activity Recognition. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Tello, A., Degeler, V., and Lazovik, A. (2023). Too Good to Be True: Performance overestimation in (re)current practices for Human Activity Recognition. arXiv.","DOI":"10.1109\/PerComWorkshops59983.2024.10503465"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_15","unstructured":"Yang, C., An, Z., Zhu, H., Hu, X., Zhang, K., Xu, K., Li, C., and Xu, Y. (2020, January 7\u201312). Gated convolutional networks with hybrid connectivity for image classification. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA."},{"key":"ref_16","unstructured":"Reiss, A. PAMAP2 Physical Activity Monitoring [Dataset]. UCI Machine Learning Repository., 2012."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1016\/S0898-1221(98)00101-1","article-title":"Interpolation methods comparison","volume":"35","author":"Caruso","year":"1998","journal-title":"Comput. Math. Appl."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1109\/MCSE.2007.55","article-title":"Matplotlib: A 2D graphics environment","volume":"9","author":"Hunter","year":"2007","journal-title":"Comput. Sci. Eng."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"133982","DOI":"10.1109\/ACCESS.2020.3010715","article-title":"Deep Neural Networks for Human Activity Recognition with Wearable Sensors: Leave-One-Subject-Out Cross-Validation for Model Selection","volume":"8","author":"Gholamiangonabadi","year":"2020","journal-title":"IEEE Access"},{"key":"ref_20","first-page":"4","article-title":"Human Activity Recognition: A Comparison of Machine Learning Approaches","volume":"2021","author":"Ambati","year":"2021","journal-title":"J. Midwest Assoc. Inf. Syst. (JMWAIS)"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2750","DOI":"10.1109\/TBME.2015.2422751","article-title":"From Wearable Sensors to Smart Implants Toward Pervasive and Personalized Healthcare","volume":"62","author":"Leff","year":"2015","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Asghari, P., and Nazerfard, E. (2018, January 17\u201319). Activity Recognition Using Hierarchical Hidden Markov Models on Streaming Sensor Data. Proceedings of the 2018 9th International Symposium on Telecommunications (IST), Tehran, Iran.","DOI":"10.1109\/ISTEL.2018.8661053"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Baldominos, A., Isasi, P., and Saez, Y. (2017, January 5\u20138). Feature selection for physical activity recognition using genetic algorithms. Proceedings of the 2017 IEEE Congress on Evolutionary Computation (CEC), Donostia, Spain. Available online: https:\/\/ieeexplore.ieee.org\/document\/7969569.","DOI":"10.1109\/CEC.2017.7969569"},{"key":"ref_24","unstructured":"Waskom, M., Botvinnik, O., O\u2019Kane, D., Hobson, P., Lukauskas, S., Gemperline, D.C., Augspurger, T., Halchenko, Y., Cole, J.B., and Warmenhoven, J. (2023, December 05). Mwaskom\/Seaborn, Available online: https:\/\/doi.org\/10.5281\/zenodo.883859."},{"key":"ref_25","unstructured":"Scikit-learn.org (2023, December 05). 1. Supervised Learning\u2014Scikit-Learn 0.21.3 Documentation. Available online: https:\/\/scikit-learn.org\/stable\/supervised_learning.html#supervised-learning."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Dargie, W. (2009, January 3\u20136). Analysis of Time and Frequency Domain Features of Accelerometer Measurements. Proceedings of the 18th International Conference on Computer Communications and Networks, San Francisco, CA, USA. Available online: https:\/\/ieeexplore.ieee.org\/document\/5235366.","DOI":"10.1109\/ICCCN.2009.5235366"},{"key":"ref_27","unstructured":"Aguirre, P.L., Torres, L.A., and Lemos, A.P. (November, January 30). Autoregressive modeling of wrist attitude for feature enrichment in human activity recognition. Proceedings of the Congresso Brasileiro de Intelig\u00eancia Computacional, Niter\u00f3i, RJ, Brazil."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"6474","DOI":"10.3390\/s140406474","article-title":"Window Size Impact in Human Activity Recognition","volume":"14","author":"Banos","year":"2014","journal-title":"Sensors"},{"key":"ref_29","unstructured":"Bock, M., H\u00f6lzemann, A., Moeller, M., and Laerhoven, K. (2023, December 05). Tutorial on Deep Learning for Human Activity Recognition. Available online: https:\/\/arxiv.org\/pdf\/2110.06663.pdf."},{"key":"ref_30","unstructured":"(2023, December 05). Module 4: Frequency Domain Signal Processing and Analysis. Available online: https:\/\/www.ccee.ncsu.edu\/ccli-sensors\/wp-content\/uploads\/sites\/4\/2015\/09\/Module-4.pdf."},{"key":"ref_31","unstructured":"Feng, Z., Mo, L., and Li, M. (2015, January 25\u201329). A Random Forest-based Ensemble Method for Activity Recognition. Proceedings of the 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1192","DOI":"10.1109\/SURV.2012.110112.00192","article-title":"A Survey on Human Activity Recognition using Wearable Sensors","volume":"15","author":"Lara","year":"2013","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Gupta, A., Gupta, K., Gupta, K., and Gupta, K. (2020, January 28\u201330). A Survey on Human Activity Recognition and Classification. Proceedings of the 2020 International Conference on Communication and Signal Processing (ICCSP), Chennai, India. Available online: https:\/\/ieeexplore.ieee.org\/document\/9182416.","DOI":"10.1109\/ICCSP48568.2020.9182416"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"5262","DOI":"10.1109\/ACCESS.2017.2684913","article-title":"Improving Activity Recognition Accuracy in Ambient-Assisted Living Systems by Automated Feature Engineering","volume":"5","author":"Zdravevski","year":"2017","journal-title":"IEEE Access"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1169","DOI":"10.1109\/JSEN.2017.2782492","article-title":"Human Activity Classification in Smartphones Using Accelerometer and Gyroscope Sensors","volume":"18","author":"Jain","year":"2018","journal-title":"IEEE Sens. J."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"e11201","DOI":"10.2196\/11201","article-title":"Applying multivariate segmentation methods to human activity recognition from wearable sensors\u2019 data","volume":"7","author":"Li","year":"2019","journal-title":"JMIR Mhealth Uhealth"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Quigley, B., Donnelly, M., Moore, G., and Galway, L. (2018). A Comparative Analysis of Windowing Approaches in Dense Sensing Environments. Proceedings, 2.","DOI":"10.3390\/proceedings2191245"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"32066","DOI":"10.1109\/ACCESS.2020.2973425","article-title":"On the Personalization of Classification Models for Human Activity Recognition","volume":"8","author":"Ferrari","year":"2020","journal-title":"IEEE Access"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1016\/j.pmcj.2017.01.008","article-title":"Position-aware activity recognition with wearable devices","volume":"38","author":"Sztyler","year":"2017","journal-title":"Pervasive Mob. Comput."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Lane, N.D., Xu, Y., Lu, H., Hu, S., Choudhury, T., Campbell, A.T., and Zhao, F. (2011, January 17\u201321). Enabling large-scale human activity inference on smartphones using community similarity networks (csn). Proceedings of the 13th International Conference on Ubiquitous Computing, Beijing China.","DOI":"10.1145\/2030112.2030160"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1154","DOI":"10.3390\/s100201154","article-title":"Machine Learning Methods for Classifying Human Physical Activity from On-Body Accelerometers","volume":"10","author":"Mannini","year":"2010","journal-title":"Sensors"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"939","DOI":"10.1212\/WNL.34.7.939","article-title":"Clinical diagnosis of Alzheimer\u2019s disease: Report of the NINCDS-ADRDA Work Group* under the auspices of Department of Health and Human Services Task Force on Alzheimer\u2019s Disease","volume":"34","author":"McKhann","year":"2010","journal-title":"Neurology"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Micucci, D., Mobilio, M., and Napoletano, P. (2017). UniMiB SHAR: A Dataset for Human Activity Recognition Using Acceleration Data from Smartphones. Appl. Sci., 7.","DOI":"10.20944\/preprints201706.0033.v1"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1016\/j.eswa.2018.03.056","article-title":"Deep Learning Algorithms for Human Activity Recognition Using Mobile and Wearable Sensor networks: State of the Art and Research Challenges","volume":"105","author":"Nweke","year":"2018","journal-title":"Expert Syst. Appl."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"717","DOI":"10.1016\/j.pmcj.2011.06.004","article-title":"Centinela: A human activity recognition system based on acceleration and vital sign data","volume":"8","author":"Lara","year":"2012","journal-title":"Pervasive Mob. Comput."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Perez, A.J., and Zeadally, S. (2021). Recent Advances in Wearable Sensing Technologies. Sensors, 21.","DOI":"10.3390\/s21206828"},{"key":"ref_47","unstructured":"Reyes-Ortiz, J.L., Anguita, D., Ghio, A., and Parra, X. (2023, December 05). Human Activity Recognition Using Smartphones Dataset. Available online: https:\/\/archive.ics.uci.edu\/dataset\/240\/human+activity+recognition+using+smartphones."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"754","DOI":"10.1016\/j.neucom.2015.07.085","article-title":"Transition-Aware Human Activity Recognition Using Smartphones","volume":"171","author":"Oneto","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Saez, Y., Baldominos, A., and Isasi, P. (2016). A Comparison Study of Classifier Algorithms for Cross- Person Physical Activity Recognition. Sensors, 17.","DOI":"10.3390\/s17010066"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Tapia, E.M., Intille, S.S., Haskell, W., Larson, K., Wright, J., King, A., and Friedman, R. (2007, January 11\u201313). Real-Time Recognition of Physical Activities and Their Intensities Using Wireless Accelerometers and a Heart Rate Monitor. Proceedings of the 2007 11th IEEE International Symposium on Wearable Computers, Boston, MA, USA.","DOI":"10.1109\/ISWC.2007.4373774"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/17\/12\/556\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:47:27Z","timestamp":1760114847000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/17\/12\/556"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,5]]},"references-count":50,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["a17120556"],"URL":"https:\/\/doi.org\/10.3390\/a17120556","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,5]]}}}