{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,20]],"date-time":"2026-03-20T05:27:16Z","timestamp":1773984436850,"version":"3.50.1"},"reference-count":44,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2019,1,25]],"date-time":"2019-01-25T00:00:00Z","timestamp":1548374400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100008530","name":"European Regional Development Fund","doi-asserted-by":"publisher","award":["NORTE-01-0145-FEDER-000026"],"award-info":[{"award-number":["NORTE-01-0145-FEDER-000026"]}],"id":[{"id":"10.13039\/501100008530","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Modern smartphones and wearables often contain multiple embedded sensors which generate significant amounts of data. This information can be used for body monitoring-based areas such as healthcare, indoor location, user-adaptive recommendations and transportation. The development of Human Activity Recognition (HAR) algorithms involves the collection of a large amount of labelled data which should be annotated by an expert. However, the data annotation process on large datasets is expensive, time consuming and difficult to obtain. The development of a HAR approach which requires low annotation effort and still maintains adequate performance is a relevant challenge. We introduce a Semi-Supervised Active Learning (SSAL) based on Self-Training (ST) approach for Human Activity Recognition to partially automate the annotation process, reducing the annotation effort and the required volume of annotated data to obtain a high performance classifier. Our approach uses a criterion to select the most relevant samples for annotation by the expert and propagate their label to the most confident samples. We present a comprehensive study comparing supervised and unsupervised methods with our approach on two datasets composed of daily living activities. The results showed that it is possible to reduce the required annotated data by more than 89% while still maintaining an accurate model performance.<\/jats:p>","DOI":"10.3390\/s19030501","type":"journal-article","created":{"date-parts":[[2019,1,25]],"date-time":"2019-01-25T11:30:00Z","timestamp":1548415800000},"page":"501","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":37,"title":["A Semi-Automatic Annotation Approach for Human Activity Recognition"],"prefix":"10.3390","volume":"19","author":[{"given":"Patr\u00edcia","family":"Bota","sequence":"first","affiliation":[{"name":"Associa\u00e7\u00e3o Fraunhofer Portugal Research, Rua Alfredo Allen 455\/461, 4200-135 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6214-5868","authenticated-orcid":false,"given":"Joana","family":"Silva","sequence":"additional","affiliation":[{"name":"Associa\u00e7\u00e3o Fraunhofer Portugal Research, Rua Alfredo Allen 455\/461, 4200-135 Porto, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8481-6079","authenticated-orcid":false,"given":"Duarte","family":"Folgado","sequence":"additional","affiliation":[{"name":"Associa\u00e7\u00e3o Fraunhofer Portugal Research, Rua Alfredo Allen 455\/461, 4200-135 Porto, Portugal"}]},{"given":"Hugo","family":"Gamboa","sequence":"additional","affiliation":[{"name":"Associa\u00e7\u00e3o Fraunhofer Portugal Research, Rua Alfredo Allen 455\/461, 4200-135 Porto, Portugal"},{"name":"Laborat\u00f3rio de Instrumenta\u00e7\u00e3o, Engenharia Biom\u00e9dica e F\u00edsica da Radia\u00e7\u00e3o (LIBPhys-UNL), Departamento de F\u00edsica, Faculdade de Ci\u00eancias e Tecnologia, FCT, Universidade Nova de Lisboa, 2829-516 Caparica, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2019,1,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1186\/1743-0003-9-21","article-title":"A review of wearable sensors and systems with application in rehabilitation","volume":"9","author":"Patel","year":"2012","journal-title":"J. NeuroEng. Rehabil."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"e1254","DOI":"10.1002\/widm.1254","article-title":"Recent trends in machine learning for human activity recognition\u2014A survey","volume":"8","author":"Roy","year":"2018","journal-title":"Wiley Interdiscip. Rev. Data Min. Knowl. Discov."},{"key":"ref_3","unstructured":"Organisation, W.H. (2018, August 19). Global Strategy on Diet, Physical Activity and Health. Available online: http:\/\/www.who.int\/dietphysicalactivity\/pa\/en\/."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"18901","DOI":"10.3390\/s150818901","article-title":"Tracking the Evolution of Smartphone Sensing for Monitoring Human Movement","volume":"15","author":"Redmond","year":"2015","journal-title":"Sensors"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1192","DOI":"10.1109\/SURV.2012.110112.00192","article-title":"A Survey on Human Activity Recognition using Wearable Sensors","volume":"15","author":"Lara","year":"2013","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"19307","DOI":"10.3390\/s141019307","article-title":"Automatic Crack Detection and Classification Method for Subway Tunnel Safety Monitoring","volume":"14","author":"Zhang","year":"2014","journal-title":"Sensors"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"33:1","DOI":"10.1145\/2499621","article-title":"A Tutorial on Human Activity Recognition Using Body-worn Inertial Sensors","volume":"46","author":"Bulling","year":"2014","journal-title":"ACM Comput. Surv."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. (2016, January 27\u201330). The Cityscapes Dataset for Semantic Urban Scene Understanding. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.350"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"31314","DOI":"10.3390\/s151229858","article-title":"Physical Human Activity Recognition Using Wearable Sensors","volume":"15","author":"Attal","year":"2015","journal-title":"Sensors"},{"key":"ref_10","first-page":"101","article-title":"Human activity classification with inertial sensors","volume":"200","author":"Silva","year":"2014","journal-title":"Stud. Health Technol. Inform."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Figueira, C., Matias, R., and Gamboa, H. (2016, January 21\u201323). Body Location Independent Activity Monitoring. Proceedings of the BIOSIGNALS, Rome, Italy.","DOI":"10.5220\/0005699601900197"},{"key":"ref_12","unstructured":"Romera-Paredes, B., Aung, M., and Bianchi-Berthouze, N. (2013, January 24\u201326). A One-vs-One Classifier Ensemble with Majority Voting for Activity Recognition. Proceedings of the ESANN 2013 proceedings 21st European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Bruges, Belgium."},{"key":"ref_13","unstructured":"Anguita, D., Ghio, A., Oneto, L., Parra, X., and Reyes-Ortiz, J.L. (2013, January 24\u201326). A Public Domain Dataset for Human Activity Recognition using Smartphones. Proceedings of the ESANN 2013 proceedings 21st European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Bruges, Belgium."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"204","DOI":"10.1016\/j.ipm.2014.07.008","article-title":"Human activity data discovery from triaxial accelerometer sensor: Non-supervised learning sensitivity to feature extraction parametrization","volume":"51","author":"Machado","year":"2015","journal-title":"Inf. Process. Manag."},{"key":"ref_15","first-page":"76","article-title":"Human Activity Recognition Based on Novel Accelerometry Features and Hidden Markov Models Application","volume":"Volume 4","author":"Gamboa","year":"2015","journal-title":"Proceedings of the International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2015)"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Murad, A., and Pyun, J.Y. (2017). Deep Recurrent Neural Networks for Human Activity Recognition. Sensors, 17.","DOI":"10.3390\/s17112556"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Ord\u00f3\u00f1ez, F.J., and Roggen, D. (2016). Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition. Sensors, 16.","DOI":"10.3390\/s16010115"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Li, F., Shirahama, K., Nisar, M.A., K\u00f6ping, L., and Grzegorzek, M. (2018). Comparison of Feature Learning Methods for Human Activity Recognition Using Wearable Sensors. Sensors, 18.","DOI":"10.3390\/s18020679"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2521","DOI":"10.1109\/TPAMI.2011.36","article-title":"Weakly Supervised Recognition of Daily Life Activities with Wearable Sensors","volume":"33","author":"Stikic","year":"2011","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"312","DOI":"10.1016\/j.pmcj.2016.08.017","article-title":"Active learning enabled activity recognition","volume":"38","author":"Hossain","year":"2017","journal-title":"Pervasive Mob. Comput."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Liu, R., Chen, T., and Huang, L. (2010, January 11\u201314). Research on human activity recognition based on active learning. Proceedings of the 2010 International Conference on Machine Learning and Cybernetics, Qingdao, China.","DOI":"10.1109\/ICMLC.2010.5581050"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Cruciani, F., Cleland, I., Nugent, C., McCullagh, P., Synnes, K., and Hallberg, J. (2018). Automatic Annotation for Human Activity Recognition in Free Living Using a Smartphone. Sensors, 18.","DOI":"10.3390\/s18072203"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Tonkin, E.L., Burrows, A., Woznowski, P., Laskowski, P., Yordanova, K.Y., Twomey, N., and Craddock, I.J. (2018). Talk, Text, Tag? Understanding Self-Annotation of Smart Home Data from a User\u2019s Perspective. Sensors, 18.","DOI":"10.3390\/s18072365"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Schr\u00f6der, M., Yordanova, K., Bader, S., and Kirste, T. (2016, January 23\u201324). Tool Support for the Online Annotation of Sensor Data. Proceedings of the 3rd International Workshop on Sensor-Based Activity Recognition and Interaction ( iWOAR \u201916), Rostock, Germany.","DOI":"10.1145\/2948963.2948972"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"15861","DOI":"10.3390\/s140915861","article-title":"Evaluation of Prompted Annotation of Activity Data Recorded from a Smart Phone","volume":"14","author":"Cleland","year":"2014","journal-title":"Sensors"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Shahmohammadi, F., Hosseini, A., King, C.E., and Sarrafzadeh, M. (2017, January 17\u201319). Smartwatch Based Activity Recognition Using Active Learning. Proceedings of the Second IEEE\/ACM International Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE \u201917), Philadelphia, PA, USA.","DOI":"10.1109\/CHASE.2017.115"},{"key":"ref_27","unstructured":"Keyson, D.V., Maher, M.L., Streitz, N., Cheok, A., Augusto, J.C., Wichert, R., Englebienne, G., Aghajan, H., and Kr\u00f6se, B.J.A. (2011, January 16\u201318). Using Active Learning to Allow Activity Recognition on a Large Scale. Proceedings of the International Joint Conference on Ambient Intelligence, Amsterdam, The Netherlands."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1323","DOI":"10.1109\/TASL.2009.2033421","article-title":"Active Learning With Sampling by Uncertainty and Density for Data Annotations","volume":"18","author":"Zhu","year":"2010","journal-title":"IEEE Trans. Audio Speech Lang. Process."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.3115\/1599081.1599224","article-title":"Active Learning with Sampling by Uncertainty and Density for Word Sense Disambiguation and Text Classification","volume":"Volume 1","author":"Zhu","year":"2008","journal-title":"Proceedings of the 22nd International Conference on Computational Linguistics (COLING \u201908)"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1016\/j.knosys.2017.03.004","article-title":"An uncertainty and density based active semi-supervised learning scheme for positive unlabeled multivariate time series classification","volume":"124","author":"He","year":"2017","journal-title":"Knowl.-Based Syst."},{"key":"ref_31","unstructured":"Stikic, M., Van Laerhoven, K., and Schiele, B. (October, January 28). Exploring semi-supervised and active learning for activity recognition. Proceedings of the 12th IEEE International Symposium on Wearable Computers, Pittaburgh, PA, USA."},{"key":"ref_32","first-page":"1","article-title":"Introduction to semi-supervised learning","volume":"3","author":"Zhu","year":"2009","journal-title":"Synth. Lect. Artif. Intell. Mach. Learn."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1007\/s10115-012-0507-8","article-title":"A survey on instance selection for active learning","volume":"35","author":"Fu","year":"2013","journal-title":"Knowl. Inf. Syst."},{"key":"ref_34","unstructured":"Settles, B. (2009). Active Learning Literature Survey, University of Wisconsin\u2013Madison. Computer Sciences Technical Report 1648."},{"key":"ref_35","unstructured":"Aggarwal, C.C., Kong, X., Gu, Q., Han, J., and Yu, P.S. (2014). Active Learning: A Survey, CRC Press. Technical Report."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1016\/j.jbi.2011.11.003","article-title":"Applying active learning to assertion classification of concepts in clinical text","volume":"45","author":"Chen","year":"2012","journal-title":"J. Biomed. Inform."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Bloodgood, M., and Vijay-Shanker, K. (2009, January 4\u20135). A Method for Stopping Active Learning Based on Stabilizing Predictions and the Need for User-adjustable Stopping. Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL \u201909), Boulder, CO, USA.","DOI":"10.3115\/1596374.1596384"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1129","DOI":"10.3115\/1599081.1599223","article-title":"Multi-criteria-based Strategy to Stop Active Learning for Data Annotation","volume":"Volume 1","author":"Zhu","year":"2008","journal-title":"Proceedings of the 22nd International Conference on Computational Linguistics (COLING \u201908)"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"He, G., Duan, Y., Li, Y., Qian, T., He, J., and Jia, X. (2015, January 9\u201311). Active Learning for Multivariate Time Series Classification with Positive Unlabeled Data. Proceedings of the 2015 IEEE 27th International Conference on Tools with Artificial Intelligence (ICTAI), Vietri sul Mare, Italy.","DOI":"10.1109\/ICTAI.2015.38"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Wei, L., and Keogh, E. (2006, January 20\u201323). Semi-supervised Time Series Classification. Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD \u201906), Philadelphia, PA, USA.","DOI":"10.1145\/1150402.1150498"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"268","DOI":"10.1016\/j.patcog.2018.04.003","article-title":"Time Alignment Measurement for Time Series","volume":"81","author":"Folgado","year":"2018","journal-title":"Pattern Recognit."},{"key":"ref_42","unstructured":"M\u00fcller, M. (2007). Dynamic Time Warping, Springer."},{"key":"ref_43","unstructured":"Danka, T., and Horvath, P. (2018, November 26). modAL: A Modular Active Learning Framework for Python. Available online: https:\/\/github.com\/cosmic-cortex\/modAL."},{"key":"ref_44","unstructured":"Reyes-Ortiz, J.L., Ghio, A., Parra, X., Anguita, D., Cabestany, J., and Catal\u00e0, A. (2013, January 24\u201326). Human Activity and Motion Disorder Recognition: Towards smarter Interactive Cognitive Environments. Proceedings of the ESANN 2013 proceedings 21st European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Bruges, Belgium."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/3\/501\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:28:39Z","timestamp":1760185719000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/3\/501"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,1,25]]},"references-count":44,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2019,2]]}},"alternative-id":["s19030501"],"URL":"https:\/\/doi.org\/10.3390\/s19030501","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,1,25]]}}}