{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,20]],"date-time":"2025-12-20T21:49:53Z","timestamp":1766267393688,"version":"build-2065373602"},"reference-count":42,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2018,7,9]],"date-time":"2018-07-09T00:00:00Z","timestamp":1531094400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100010665","name":"H2020 Marie Sk\u0142odowska-Curie Actions","doi-asserted-by":"publisher","award":["676157"],"award-info":[{"award-number":["676157"]}],"id":[{"id":"10.13039\/100010665","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Data annotation is a time-consuming process posing major limitations to the development of Human Activity Recognition (HAR) systems. The availability of a large amount of labeled data is required for supervised Machine Learning (ML) approaches, especially in the case of online and personalized approaches requiring user specific datasets to be labeled. The availability of such datasets has the potential to help address common problems of smartphone-based HAR, such as inter-person variability. In this work, we present (i) an automatic labeling method facilitating the collection of labeled datasets in free-living conditions using the smartphone, and (ii) we investigate the robustness of common supervised classification approaches under instances of noisy data. We evaluated the results with a dataset consisting of 38 days of manually labeled data collected in free living. The comparison between the manually and the automatically labeled ground truth demonstrated that it was possible to obtain labels automatically with an 80\u201385% average precision rate. Results obtained also show how a supervised approach trained using automatically generated labels achieved an 84% f-score (using Neural Networks and Random Forests); however, results also demonstrated how the presence of label noise could lower the f-score up to 64\u201374% depending on the classification approach (Nearest Centroid and Multi-Class Support Vector Machine).<\/jats:p>","DOI":"10.3390\/s18072203","type":"journal-article","created":{"date-parts":[[2018,7,9]],"date-time":"2018-07-09T11:18:53Z","timestamp":1531135133000},"page":"2203","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":48,"title":["Automatic Annotation for Human Activity Recognition in Free Living Using a Smartphone"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1870-0203","authenticated-orcid":false,"given":"Federico","family":"Cruciani","sequence":"first","affiliation":[{"name":"Computer Science Research Institute, Ulster University, Newtownabbey BT370QB, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2368-7354","authenticated-orcid":false,"given":"Ian","family":"Cleland","sequence":"additional","affiliation":[{"name":"Computer Science Research Institute, Ulster University, Newtownabbey BT370QB, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0882-7902","authenticated-orcid":false,"given":"Chris","family":"Nugent","sequence":"additional","affiliation":[{"name":"Computer Science Research Institute, Ulster University, Newtownabbey BT370QB, UK"}]},{"given":"Paul","family":"McCullagh","sequence":"additional","affiliation":[{"name":"Computer Science Research Institute, Ulster University, Newtownabbey BT370QB, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4549-6751","authenticated-orcid":false,"given":"K\u00e5re","family":"Synnes","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Electrical and Space Engineering, Lule\u00e5 University of Technology, 97187 Lule\u00e5, Sweden"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3191-8335","authenticated-orcid":false,"given":"Josef","family":"Hallberg","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Electrical and Space Engineering, Lule\u00e5 University of Technology, 97187 Lule\u00e5, Sweden"}]}],"member":"1968","published-online":{"date-parts":[[2018,7,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"388","DOI":"10.1016\/j.bbe.2017.04.004","article-title":"Physical activity recognition by smartphones, a survey","volume":"37","author":"Morales","year":"2017","journal-title":"Biocybern. Biomed. Eng."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2059","DOI":"10.3390\/s150102059","article-title":"A survey of online activity recognition using mobile phones","volume":"15","author":"Shoaib","year":"2015","journal-title":"Sensors"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1109\/TITB.2007.899496","article-title":"Detection of daily activities and sports with wearable sensors in controlled and uncontrolled conditions","volume":"12","author":"Ermes","year":"2008","journal-title":"IEEE Trans. Inf. Technol. Biomed."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1007\/s12668-013-0088-3","article-title":"A Review and Taxonomy of Activity Recognition on Mobile Phones","volume":"3","author":"Incel","year":"2013","journal-title":"BioNanoScience"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Schr\u00f6der, M., Yordanova, K., Bader, S., and Kirste, T. (2016, January 23\u201324). Tool support for the online annotation of sensor data. Proceedings of the 3rd International Workshop on Sensor-based Activity Recognition and Interaction, Rostock, Germany.","DOI":"10.1145\/2948963.2948972"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2499621","article-title":"A tutorial on human activity recognition using body-worn inertial sensors","volume":"1","author":"Bulling","year":"2014","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1192","DOI":"10.1109\/SURV.2012.110112.00192","article-title":"A Survey on Human Activity Recognition using Wearable Sensors","volume":"15","author":"Lara","year":"2013","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Cruciani, F., Donnelly, M.P., Nugent, C.D., Parente, G., Paggetti, C., and Burns, W. (2010, January 13\u201315). DANTE: A video based annotation tool for smart environments. Proceedings of the International Conference on Sensor Systems and Software, Miami, FL, USA.","DOI":"10.1007\/978-3-642-23583-2_13"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Woznowski, P., Tonkin, E., Laskowski, P., Twomey, N., Yordanova, K., and Burrows, A. (2017, January 13\u201317). Talk, text or tag?: The development of a self-annotation app for activity recognition in smart environments. Proceedings of the IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops, Big Island, HI, USA.","DOI":"10.1109\/PERCOMW.2017.7917544"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"15861","DOI":"10.3390\/s140915861","article-title":"Evaluation of prompted annotation of activity data recorded from a smart phone","volume":"14","author":"Cleland","year":"2014","journal-title":"Sensors"},{"key":"ref_11","unstructured":"Yordanova, K., Paiement, A., Schr\u00f6der, M., Tonkin, E., Woznowski, P., Olsson, C.M., Rafferty, J., and Sztyler, T. (arXiv, 2018). Challenges in Annotation of useR Data for UbiquitOUs Systems: Results from the 1st ARDUOUS Workshop, arXiv."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2521","DOI":"10.1109\/TPAMI.2011.36","article-title":"Weakly Supervised Recognition of Daily Life Activities with Wearable Sensors","volume":"33","author":"Stikic","year":"2011","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"3115","DOI":"10.1016\/j.eswa.2014.11.063","article-title":"Extending knowledge-driven activity models through data-driven learning techniques","volume":"42","author":"Azkune","year":"2015","journal-title":"Expert Syst. Appl."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Cruciani, F., Nugent, C., Cleland, I., McCullagh, P., Synnes, K., and Hallberg, J. (2018, January 19\u201323). Personalized Online Training for Physical Activity Monitoring using Weak Labels. Proceedings of the IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), Athens, Greece.","DOI":"10.1109\/PERCOMW.2018.8480292"},{"key":"ref_15","first-page":"38","article-title":"Recognizing Human Activities User-independently on Smartphones Based on Accelerometer Data","volume":"1","author":"Siirtola","year":"2012","journal-title":"Int. J. Interact. Multimed. Artif. Intelligence"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1245","DOI":"10.1088\/0967-3334\/35\/7\/1245","article-title":"Activity classification based on inertial and barometric pressure sensors at different anatomical locations","volume":"35","author":"Leuenberger","year":"2014","journal-title":"Physiol. Meas."},{"key":"ref_17","unstructured":"Dernbach, S., Das, B., Krishnan, N.C., Thomas, B.L., and Cook, D.J. (2017, January 26\u201329). Simple and Complex Activity Recognition through Smart Phones. Proceedings of the IEEE Eighth International Conference on Intelligent Environments, Guanajuato, Mexico."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"675","DOI":"10.1007\/s00779-012-0515-4","article-title":"Activity logging using lightweight classification techniques in mobile devices","volume":"17","author":"Bernardos","year":"2013","journal-title":"Pers. Ubiquit. Comput."},{"key":"ref_19","unstructured":"Kose, M., Incel, O.D., and Ersoy, C. (2012, January 6). Online human activity recognition on smart phones. Proceedings of the Workshop on Mobile Sensing: From Smartphones and Wearables to Big Data, Beijing, China."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Janidarmian, M., Fekr, A.R., Radecka, K., and Zilic, Z. (2017). A comprehensive analysis on wearable acceleration sensors in human activity recognition. Sensors, 17.","DOI":"10.3390\/s17030529"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1109\/TST.2014.6838194","article-title":"Activity recognition with smartphone sensors","volume":"19","author":"Su","year":"2014","journal-title":"Tsinghua Sci. Technol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1109\/MPRV.2010.7","article-title":"Human activity recognition and pattern discovery","volume":"9","author":"Kim","year":"2010","journal-title":"IEEE Perv. Comput."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Lee, Y.S., and Cho, S.B. (2011, January 23\u201325). Activity Recognition Using Hierarchical Hidden Markov Models on a Smartphone with 3D Accelerometer. Proceedings of  the 6th International Conference on Hybrid Artificial Intelligence Systems, Wroclaw, Poland.","DOI":"10.1007\/978-3-642-21219-2_58"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1016\/j.eswa.2016.04.032","article-title":"Human activity recognition with smartphone sensors using deep learning neural networks","volume":"59","author":"Ronao","year":"2016","journal-title":"Expert Syst. Appl."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Ord\u00f3\u00f1ez, F.J., and Roggen, D. (2016). Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors, 16.","DOI":"10.3390\/s16010115"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1016\/j.future.2017.11.029","article-title":"A robust human activity recognition system using smartphone sensors and deep learning","volume":"81","author":"Hassan","year":"2018","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_27","first-page":"25","article-title":"Handling imbalanced datasets : A review","volume":"30","author":"Kotsiantis","year":"2006","journal-title":"Science"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"731","DOI":"10.3758\/BRM.41.3.731","article-title":"The observer XT: A tool for the integration and synchronization of multimodal signals","volume":"41","author":"Zimmerman","year":"2009","journal-title":"Behav. Res. Methods"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1109\/TCYB.2015.2396291","article-title":"Pervasive Sound Sensing: A Weakly Supervised Training Approach","volume":"46","author":"Kelly","year":"2016","journal-title":"IEEE Trans. Cybern."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Buehler, P., Everingham, M., and Zisserman, A. (2009, January 20\u201325). Learning sign language by watching TV (using weakly aligned subtitles). Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPR Workshops 2009), Miami, FL, USA.","DOI":"10.1109\/CVPRW.2009.5206523"},{"key":"ref_31","first-page":"1295","article-title":"Energy efficient smartphone-based activity recognition using fixed-point arithmetic","volume":"19","author":"Anguita","year":"2013","journal-title":"J. Univers. Comput. Sci."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1402","DOI":"10.3390\/s130201402","article-title":"Human behavior cognition using smartphone sensors","volume":"13","author":"Pei","year":"2013","journal-title":"Sensors"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1016\/j.procs.2014.07.009","article-title":"A study on human activity recognition using accelerometer data from smartphones","volume":"34","author":"Bayat","year":"2014","journal-title":"Procedia Comput. Sci."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"242","DOI":"10.1016\/j.pmcj.2014.05.006","article-title":"Using unlabeled data in a sparse-coding framework for human activity recognition","volume":"15","author":"Bhattacharya","year":"2014","journal-title":"Perv. Mobile Comput."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"754","DOI":"10.1016\/j.neucom.2015.07.085","article-title":"Transition-Aware Human Activity Recognition Using Smartphones","volume":"171","author":"Oneto","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1109\/THMS.2015.2489688","article-title":"Toward personalized activity recognition systems with a semipopulation approach","volume":"46","author":"Hong","year":"2016","journal-title":"IEEE Trans. Hum.-Mach. Syst."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1016\/j.jpdc.2017.05.007","article-title":"GCHAR: An efficient Group-based Context\u2014Aware human activity recognition on smartphone","volume":"118","author":"Cao","year":"2018","journal-title":"J. Parallel Distrib. Comput."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1016\/j.engappai.2018.04.002","article-title":"Robust Human Activity Recognition using smartwatches and smartphones","volume":"72","author":"Blunck","year":"2018","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"410","DOI":"10.1016\/j.amepre.2009.01.021","article-title":"Translating Physical Activity Recommendations into a Pedometer-Based Step Goal","volume":"36","author":"Marshall","year":"2009","journal-title":"Am. J. Prev. Med."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Cruciani, F., Nugent, C., Cleland, I., and McCullagh, P. (2017, January 11\u201315). Rich Context Information for Just-In-Time Adaptive Intervention promoting physical activity. Proceedings of 39th Annual International Conference of the IEEE on the Engineering in Medicine and Biology Society (EMBC), Seogwipo, Korea.","DOI":"10.1109\/EMBC.2017.8036957"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1113\/jphysiol.1991.sp018586","article-title":"The two power limits conditioning step frequency in human running","volume":"437","author":"Cavagna","year":"1991","journal-title":"J. Physiol."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"290","DOI":"10.1016\/j.ssci.2015.07.027","article-title":"The German Naturalistic Cycling Study\u2014Comparing cycling speed of riders of different e-bikes and conventional bicycles","volume":"92","author":"Schleinitz","year":"2017","journal-title":"Saf. Sci."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/7\/2203\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:11:46Z","timestamp":1760195506000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/7\/2203"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,7,9]]},"references-count":42,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2018,7]]}},"alternative-id":["s18072203"],"URL":"https:\/\/doi.org\/10.3390\/s18072203","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2018,7,9]]}}}