{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,13]],"date-time":"2026-03-13T09:12:16Z","timestamp":1773393136430,"version":"3.50.1"},"reference-count":32,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2016,12,11]],"date-time":"2016-12-11T00:00:00Z","timestamp":1481414400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The popularity of using wearable inertial sensors for physical activity classification has dramatically increased in the last decade due to their versatility, low form factor, and low power requirements. Consequently, various systems have been developed to automatically classify daily life activities. However, the scope and implementation of such systems is limited to laboratory-based investigations. Furthermore, these systems are not directly comparable, due to the large diversity in their design (e.g., number of sensors, placement of sensors, data collection environments, data processing techniques, features set, classifiers, cross-validation methods). Hence, the aim of this study is to propose a fair and unbiased benchmark for the field-based validation of three existing systems, highlighting the gap between laboratory and real-life conditions. For this purpose, three representative state-of-the-art systems are chosen and implemented to classify the physical activities of twenty older subjects (76.4 \u00b1 5.6 years). The performance in classifying four basic activities of daily life (sitting, standing, walking, and lying) is analyzed in controlled and free living conditions. To observe the performance of laboratory-based systems in field-based conditions, we trained the activity classification systems using data recorded in a laboratory environment and tested them in real-life conditions in the field. The findings show that the performance of all systems trained with data in the laboratory setting highly deteriorates when tested in real-life conditions, thus highlighting the need to train and test the classification systems in the real-life setting. Moreover, we tested the sensitivity of chosen systems to window size (from 1 s to 10 s) suggesting that overall accuracy decreases with increasing window size. Finally, to evaluate the impact of the number of sensors on the performance, chosen systems are modified considering only the sensing unit worn at the lower back. The results, similarly to the multi-sensor setup, indicate substantial degradation of the performance when laboratory-trained systems are tested in the real-life setting. This degradation is higher than in the multi-sensor setup. Still, the performance provided by the single-sensor approach, when trained and tested with real data, can be acceptable (with an accuracy above 80%).<\/jats:p>","DOI":"10.3390\/s16122105","type":"journal-article","created":{"date-parts":[[2016,12,12]],"date-time":"2016-12-12T14:43:57Z","timestamp":1481553837000},"page":"2105","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":30,"title":["Performance Evaluation of State of the Art Systems for Physical Activity Classification of Older Subjects Using Inertial Sensors in a Real Life Scenario: A Benchmark Study"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6421-9245","authenticated-orcid":false,"given":"Muhammad","family":"Awais","sequence":"first","affiliation":[{"name":"Department of Electrical, Electronic, and Information Engineering Guglielmo Marconi, University of Bologna, Bologna 40126, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Luca","family":"Palmerini","sequence":"additional","affiliation":[{"name":"Department of Electrical, Electronic, and Information Engineering Guglielmo Marconi, University of Bologna, Bologna 40126, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7111-422X","authenticated-orcid":false,"given":"Alan","family":"Bourke","sequence":"additional","affiliation":[{"name":"Department of Neuroscience, Norwegian University of Science and Technology, Trondheim, Norway"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Espen","family":"Ihlen","sequence":"additional","affiliation":[{"name":"Department of Neuroscience, Norwegian University of Science and Technology, Trondheim, Norway"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0214-9290","authenticated-orcid":false,"given":"Jorunn","family":"Helbostad","sequence":"additional","affiliation":[{"name":"Department of Neuroscience, Norwegian University of Science and Technology, Trondheim, Norway"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2318-4370","authenticated-orcid":false,"given":"Lorenzo","family":"Chiari","sequence":"additional","affiliation":[{"name":"Department of Electrical, Electronic, and Information Engineering Guglielmo Marconi, University of Bologna, Bologna 40126, Italy"},{"name":"Health Sciences and Technologies Interdepartmental Center for Industrial Research, University of Bologna, Bologna 40126, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2016,12,11]]},"reference":[{"key":"ref_1","unstructured":"World Health Organization for Europe: Policies and Priority Interventions for Healthy Ageing. Available online: http:\/\/www.euro.who.int\/__data\/assets\/pdf_file\/0006\/161637\/WHD-Policies-and-Priority-Interventions-for-Healthy-Ageing.pdf?ua=1."},{"key":"ref_2","unstructured":"Bao, L., and Intille, S.S. (2004). International Conference on Pervasive Computing, Springer."},{"key":"ref_3","unstructured":"Ravi, N., Dandekar, N., Mysore, P., and Littman, M.L. (2005, January 9\u201313). Activity recognition from accelerometer data. Proceedings of the Twentieth National Conference on Artificial Intelligence, Pittsburgh, PA, USA."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"871","DOI":"10.1109\/TBME.2008.2006190","article-title":"A comparison of feature extraction methods for the classification of dynamic activities from accelerometer data","volume":"56","author":"Preece","year":"2009","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"3605","DOI":"10.1016\/j.patcog.2010.04.019","article-title":"Comparative study on classifying human activities with miniature inertial and magnetic sensors","volume":"43","author":"Altun","year":"2010","journal-title":"Pattern Recognit."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1166","DOI":"10.1109\/TITB.2010.2051955","article-title":"A triaxial accelerometer-based physical-activity recognition via augmented-signal features and a hierarchical recognizer","volume":"14","author":"Khan","year":"2010","journal-title":"IEEE Trans. Inf. Technol. Biomed."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1154","DOI":"10.3390\/s100201154","article-title":"Machine learning methods for classifying human physical activity from on-body accelerometers","volume":"10","author":"Mannini","year":"2010","journal-title":"Sensors"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2138","DOI":"10.1249\/MSS.0b013e31825e825a","article-title":"Computational methods for estimating energy expenditure in human physical activities","volume":"44","author":"Liu","year":"2012","journal-title":"Med. Sci. Sports Exerc."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"9183","DOI":"10.3390\/s130709183","article-title":"Optimal placement of accelerometers for the detection of everyday activities","volume":"13","author":"Cleland","year":"2013","journal-title":"Sensors"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Leutheuser, H., Schuldhaus, D., and Eskofier, B.M. (2013). Hierarchical, multi-sensor based classification of daily life activities: Comparison with state-of-the-art algorithms using a benchmark dataset. PLoS ONE, 8.","DOI":"10.1371\/journal.pone.0075196"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2269","DOI":"10.1088\/0967-3334\/35\/11\/2269","article-title":"A comparison of activity classification in younger and older cohorts using a smartphone","volume":"35","author":"Rosario","year":"2014","journal-title":"Physiol. Meas."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2191","DOI":"10.1088\/0967-3334\/35\/11\/2191","article-title":"A random forest classifier for the prediction of energy expenditure and type of physical activity from wrist and hip accelerometers","volume":"35","author":"Ellis","year":"2014","journal-title":"Physiol. Meas."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"779","DOI":"10.1016\/j.medengphy.2014.02.012","article-title":"Evaluation of accelerometer based multi-sensor versus single-sensor activity recognition systems","volume":"36","author":"Gao","year":"2014","journal-title":"Med. Eng. Phys."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"670","DOI":"10.1016\/j.medengphy.2014.02.009","article-title":"Activity recognition with smartphone support","volume":"36","author":"Guiry","year":"2014","journal-title":"Med. Eng. Phys."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zheng, Y. (2015). Human activity recognition based on the hierarchical feature selection and classification framework. J. Electr. Comput. Eng., 2015.","DOI":"10.1155\/2015\/140820"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/j.gaitpost.2015.11.005","article-title":"Comparison of physical activity assessed using hip-and wrist-worn accelerometers","volume":"44","author":"Kamada","year":"2016","journal-title":"Gait Posture"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"829","DOI":"10.1109\/TASE.2013.2256349","article-title":"An unsupervised approach for automatic activity recognition based on hidden Markov model regression","volume":"10","author":"Trabelsi","year":"2013","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1245","DOI":"10.1088\/0967-3334\/35\/7\/1245","article-title":"Activity classification based on inertial and barometric pressure sensors at different anatomical locations","volume":"35","author":"Torres","year":"2014","journal-title":"Physiol. Meas."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"304","DOI":"10.1007\/BF02513304","article-title":"Physical activity monitoring based on accelerometry: Validation and comparison with video observation","volume":"37","author":"Aminian","year":"1999","journal-title":"Med. Biol. Eng. Comput."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"645","DOI":"10.1007\/s00779-010-0293-9","article-title":"Preprocessing techniques for context recognition from accelerometer data","volume":"14","author":"Figo","year":"2010","journal-title":"Pers. Ubiquitous Comput."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"705","DOI":"10.1016\/j.medengphy.2015.04.005","article-title":"Varying behavior of different window sizes on the classification of static and dynamic physical activities from a single accelerometer","volume":"37","author":"Fida","year":"2015","journal-title":"Med. Eng. Phys."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Shoaib, M., Bosch, S., Incel, O.D., Scholten, H., and Havinga, P.J. (2016). Complex Human Activity Recognition Using Smartphone and Wrist-Worn Motion Sensors. Sensors, 16.","DOI":"10.3390\/s16040426"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"711","DOI":"10.1109\/TBME.2003.812189","article-title":"Ambulatory system for human motion analysis using a kinematic sensor: Monitoring of daily physical activity in the elderly","volume":"50","author":"Najafi","year":"2003","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"497","DOI":"10.1016\/j.medengphy.2004.11.006","article-title":"A description of an accelerometer-based mobility monitoring technique","volume":"27","author":"Lyons","year":"2005","journal-title":"Med. Eng. Phys."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1127","DOI":"10.1016\/j.medengphy.2011.05.002","article-title":"Activity classification using a single chest mounted tri-axial accelerometer","volume":"33","author":"Godfrey","year":"2011","journal-title":"Med. Eng. Phys."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"688","DOI":"10.1109\/TNSRE.2012.2202691","article-title":"Detection and classification of postural transitions in real-world conditions","volume":"20","author":"Ganea","year":"2012","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Awais, M., Mellone, S., and Chiari, L. (2015, January 25\u201329). Physical activity classification meets daily life: Review on existing methodologies and open challenges. Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy.","DOI":"10.1109\/EMBC.2015.7319526"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1267","DOI":"10.3390\/s140101267","article-title":"Recommendations for standardizing validation procedures assessing physical activity of older persons by monitoring body postures and movements","volume":"14","author":"Lindemann","year":"2014","journal-title":"Sensors"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2193","DOI":"10.1249\/MSS.0b013e31829736d6","article-title":"Activity recognition using a single accelerometer placed at the wrist or ankle","volume":"45","author":"Mannini","year":"2013","journal-title":"Med. Sci. Sports Exerc."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10916-014-0095-0","article-title":"Better physical activity classification using smartphone acceleration sensor","volume":"38","author":"Arif","year":"2014","journal-title":"J. Med. Syst."},{"key":"ref_31","unstructured":"Bourke, A.K., Ihlen, E.A.F., Bergquist, R., Wik, P.B., and Helbostad, J.L. (2016). Validation of Existing Systems and Development of New Algorithms for Body-Worn Activity Classification Systems for Independent Living Older Adults-Description of the Study Protocol and a Reference Dataset. Sensors, (accepted)."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1145\/1656274.1656278","article-title":"The WEKA data mining software: An update","volume":"11","author":"Hall","year":"2009","journal-title":"ACM SIGKDD Explor. Newsl."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/16\/12\/2105\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T19:28:24Z","timestamp":1760210904000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/16\/12\/2105"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,12,11]]},"references-count":32,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2016,12]]}},"alternative-id":["s16122105"],"URL":"https:\/\/doi.org\/10.3390\/s16122105","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,12,11]]}}}