{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,11]],"date-time":"2026-04-11T09:26:11Z","timestamp":1775899571433,"version":"3.50.1"},"reference-count":31,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2019,11,25]],"date-time":"2019-11-25T00:00:00Z","timestamp":1574640000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This study presents incremental learning based methods to personalize human activity recognition models. Initially, a user-independent model is used in the recognition process. When a new user starts to use the human activity recognition application, personal streaming data can be gathered. Of course, this data does not have labels. However, there are three different ways to obtain this data: non-supervised, semi-supervised, and supervised. The non-supervised approach relies purely on predicted labels, the supervised approach uses only human intelligence to label the data, and the proposed method for semi-supervised learning is a combination of these two: It uses artificial intelligence (AI) in most cases to label the data but in uncertain cases it relies on human intelligence. After labels are obtained, the personalization process continues by using the streaming data and these labels to update the incremental learning based model, which in this case is Learn++. Learn++ is an ensemble method that can use any classifier as a base classifier, and this study compares three base classifiers: linear discriminant analysis (LDA), quadratic discriminant analysis (QDA), and classification and regression tree (CART). Moreover, three datasets are used in the experiment to show how well the presented method generalizes on different datasets. The results show that personalized models are much more accurate than user-independent models. On average, the recognition rates are: 87.0% using the user-independent model, 89.1% using the non-supervised personalization approach, 94.0% using the semi-supervised personalization approach, and 96.5% using the supervised personalization approach. This means that by relying on predicted labels with high confidence, and asking the user to label only uncertain observations (6.6% of the observations when using LDA, 7.7% when using QDA, and 18.3% using CART), almost as low error rates can be achieved as by using the supervised approach, in which labeling is fully based on human intelligence.<\/jats:p>","DOI":"10.3390\/s19235151","type":"journal-article","created":{"date-parts":[[2019,11,25]],"date-time":"2019-11-25T03:10:00Z","timestamp":1574651400000},"page":"5151","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":36,"title":["Incremental Learning to Personalize Human Activity Recognition Models: The Importance of Human AI Collaboration"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5995-5421","authenticated-orcid":false,"given":"Pekka","family":"Siirtola","sequence":"first","affiliation":[{"name":"Biomimetics and Intelligent Systems Group, University of Oulu, P.O. BOX 4500, FI-90014 Oulu, Finland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9993-8602","authenticated-orcid":false,"given":"Juha","family":"R\u00f6ning","sequence":"additional","affiliation":[{"name":"Biomimetics and Intelligent Systems Group, University of Oulu, P.O. BOX 4500, FI-90014 Oulu, Finland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,11,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"158","DOI":"10.3389\/fneur.2012.00158","article-title":"Using mobile phones for activity recognition in Parkinson\u2019s patients","volume":"3","author":"Albert","year":"2012","journal-title":"Front. Neurol."},{"key":"ref_2","unstructured":"Weiss, G., and Lockhart, J. (2012, January 22\u201326). The impact of personalization on smartphone-based activity recognition. Proceedings of the AAAI Workshop on Activity Context Representation: Techniques and Languages, Toronto, ON, Canada."},{"key":"ref_3","unstructured":"Munguia Tapia, E. (2008). Using Machine Learning for Real-Time Activity Recognition and Estimation of Energy Expenditure. [Ph.D. Thesis, Massachusetts Institute of Technology]."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"33:1","DOI":"10.1145\/2499621","article-title":"A Tutorial on Human Activity Recognition Using Body-worn Inertial Sensors","volume":"46","author":"Bulling","year":"2014","journal-title":"ACM Comput. Surv."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Gong, T., Kim, Y., Shin, J., and Lee, S.J. (2019, January 10\u201313). MetaSense: Few-Shot Adaptation to Untrained Conditions in Deep Mobile Sensing. Proceedings of the 17th ACM Conference on Embedded Networked Sensor Systems, New York, NY, USA.","DOI":"10.1145\/3356250.3360020"},{"key":"ref_6","unstructured":"Siirtola, P., Koskim\u00e4ki, H., and R\u00f6ning, J. (2016, January 27\u201329). From user-independent to personal human activity recognition models using smartphone sensors. Proceedings of the European Symposium on Artificial Neural Networks (ESANN), Bruges, Belgium."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Fallahzadeh, R., and Ghasemzadeh, H. (2017, January 18\u201320). Personalization Without User Interruption: Boosting Activity Recognition in New Subjects Using Unlabeled Data. Proceedings of the 8th International Conference on Cyber-Physical Systems, Pittsburgh, PA, USA.","DOI":"10.1145\/3055004.3055015"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Garcia-Ceja, E., Riegler, M., Kvernberg, A.K., and Torresen, J. (2019). User-adaptive models for activity and emotion recognition using deep transfer learning and data augmentation. User Model. User-Adapted Interact., 1\u201329.","DOI":"10.1007\/s11257-019-09248-1"},{"key":"ref_9","unstructured":"Gepperth, A., and Hammer, B. (2016, January 27\u201329). Incremental learning algorithms and applications. Proceedings of the European Symposium on Artificial Neural Networks (ESANN), Bruges, Belgium."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"171","DOI":"10.3233\/AIS-150308","article-title":"Adapting activity recognition to a person with Multi-Classifier Adaptive Training","volume":"7","author":"Gams","year":"2015","journal-title":"J. Ambient Intel. Smart Environ."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"691","DOI":"10.1109\/TITB.2012.2196440","article-title":"An incremental learning method based on probabilistic neural networks and adjustable fuzzy clustering for human activity recognition by using wearable sensors","volume":"16","author":"Wang","year":"2012","journal-title":"IEEE Trans. Inf. Technol. Biomed."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Mo, L., Feng, Z., and Qian, J. (2016, January 11\u201313). Human daily activity recognition with wearable sensors based on incremental learning. Proceedings of the 2016 10th International Conference on Sensing Technology (ICST), Nanjing, China.","DOI":"10.1109\/ICSensT.2016.7796224"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ntalampiras, S., and Roveri, M. (2016, January 6\u20139). An incremental learning mechanism for human activity recognition. Proceedings of the IEEE Symposium Series on Computational Intelligence, Athens, Greece.","DOI":"10.1109\/SSCI.2016.7850188"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Miu, T., Missier, P., and Pl\u00f6tz, T. (2015, January 26\u201328). Bootstrapping personalised human activity recognition models using online active learning. Proceedings of the 2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing, Liverpool, UK.","DOI":"10.1109\/CIT\/IUCC\/DASC\/PICOM.2015.170"},{"key":"ref_15","unstructured":"Siirtola, P., Koskim\u00e4ki, H., and R\u00f6ning, J. (2018, January 24\u201326). Personalizing human activity recognition models using incremental learning. Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), Bruges, Belgium."},{"key":"ref_16","unstructured":"(2019, November 24). The Guardian: Tay. Available online: https:\/\/www.theguardian.com\/technology\/2016\/mar\/24\/tay-microsofts-ai-chatbot-gets-a-crash-course-in-racism-from-twitter."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"304","DOI":"10.1016\/j.neucom.2014.09.074","article-title":"Adaptive mobile activity recognition system with evolving data streams","volume":"150","author":"Abdallah","year":"2015","journal-title":"Neurocomputing"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1585","DOI":"10.1109\/JBHI.2018.2869779","article-title":"Classifier Personalization for Activity Recognition Using Wrist Accelerometers","volume":"23","author":"Mannini","year":"2019","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Siirtola, P., Koskim\u00e4ki, H., and R\u00f6ning, J. (2019, January 24\u201326). Importance of user inputs while using incremental learning to personalize human activity recognition models. Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), Bruges, Belgium.","DOI":"10.3390\/s19235151"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"10146","DOI":"10.3390\/s140610146","article-title":"Fusion of Smartphone Motion Sensors for Physical Activity Recognition","volume":"14","author":"Shoaib","year":"2014","journal-title":"Sensors"},{"key":"ref_21","first-page":"38","article-title":"Recognizing Human Activities User-independently on Smartphones Based on Accelerometer Data","volume":"1","author":"Siirtola","year":"2012","journal-title":"Int. J. Interact. Multimed. Artif. Intell."},{"key":"ref_22","unstructured":"Anguita, D., Ghio, A., Oneto, L., Parra Perez, X., and Reyes Ortiz, J.L. (2013, January 24\u201326). A public domain dataset for human activity recognition using smartphones. Proceedings of the 21th International European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning, Bruges, Belgium."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Siirtola, P., Koskim\u00e4ki, H., and R\u00f6ning, J. (2018, January 8\u201312). OpenHAR: A Matlab Toolbox for Easy Access to Publicly Open Human Activity Data Sets. Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers, Singapore.","DOI":"10.1145\/3267305.3267503"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"497","DOI":"10.1109\/5326.983933","article-title":"Learn++: An incremental learning algorithm for supervised neural networks","volume":"31","author":"Polikar","year":"2001","journal-title":"IEEE Trans. Syst. Man Cybern. Part C"},{"key":"ref_25","unstructured":"Losing, V., Hammer, B., and Wersing, H. (2016, January 27\u201329). Choosing the Best Algorithm for an Incremental On-line Learning Task. Proceedings of the European Symposium on Artificial Neural Networks, Bruges, Belgium."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Siirtola, P., Koskim\u00e4ki, H., and R\u00f6ning, J. (2011, January 11\u201315). Periodic quick test for classifying long-term activities. Proceedings of the 2011 IEEE Symposium on Computational Intelligence and Data Mining (CIDM), Paris, France.","DOI":"10.1109\/CIDM.2011.5949426"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Siirtola, P., and R\u00f6ning, J. (2016, January 24\u201326). Reducing Uncertainty in User-independent Activity Recognition-A Sensor Fusion-based Approach. Proceedings of the 5th International Conference on Pattern Recognition Applications and Methods (ICPRAM), Rome, Italy.","DOI":"10.5220\/0005743106110619"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Siirtola, P., Koskim\u00e4ki, H., and R\u00f6ning, J. (2016, January 6\u20139). Personal models for eHealth-improving user-dependent human activity recognition models using noise injection. Proceedings of the 2016 IEEE Symposium Series on the Computational Intelligence (SSCI), Athens, Greece.","DOI":"10.1109\/SSCI.2016.7849944"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Mairittha, T., Mairittha, N., and Inoue, S. (2019, January 9\u201313). A Dialogue-based Annotation for Activity Recognition. Proceedings of the Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers, London, UK.","DOI":"10.1145\/3341162.3345594"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Elwell, R., and Polikar, R. (2009, January 14\u201319). Incremental learning in nonstationary environments with controlled forgetting. Proceedings of the 2009 International Joint Conference on Neural Networks, Atlanta, GA, USA.","DOI":"10.1109\/IJCNN.2009.5178779"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"6369","DOI":"10.1109\/JSEN.2018.2845749","article-title":"Activity recognition for incomplete spinal cord injury subjects using hidden Markov models","volume":"18","author":"Sok","year":"2018","journal-title":"IEEE Sens. J."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/23\/5151\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:37:12Z","timestamp":1760189832000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/23\/5151"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,25]]},"references-count":31,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2019,12]]}},"alternative-id":["s19235151"],"URL":"https:\/\/doi.org\/10.3390\/s19235151","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,11,25]]}}}