{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:48:26Z","timestamp":1760240906444,"version":"build-2065373602"},"reference-count":28,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2019,10,17]],"date-time":"2019-10-17T00:00:00Z","timestamp":1571270400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004919","name":"King Abdulaziz City for Science and Technology","doi-asserted-by":"publisher","award":["13-ENE2356-10"],"award-info":[{"award-number":["13-ENE2356-10"]}],"id":[{"id":"10.13039\/501100004919","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>In supervised Activities of Daily Living (ADL) recognition systems, annotating collected sensor readings is an essential, yet exhaustive, task. Readings are collected from activity-monitoring sensors in a 24\/7 manner. The size of the produced dataset is so huge that it is almost impossible for a human annotator to give a certain label to every single instance in the dataset. This results in annotation gaps in the input data to the adopting learning system. The performance of the recognition system is negatively affected by these gaps. In this work, we propose and investigate three different paradigms to handle these gaps. In the first paradigm, the gaps are taken out by dropping all unlabeled readings. A single \u201cUnknown\u201d or \u201cDo-Nothing\u201d label is given to the unlabeled readings within the operation of the second paradigm. The last paradigm handles these gaps by giving every set of them a unique label identifying the encapsulating certain labels. Also, we propose a semantic preprocessing method of annotation gaps by constructing a hybrid combination of some of these paradigms for further performance improvement. The performance of the proposed three paradigms and their hybrid combination is evaluated using an ADL benchmark dataset containing more than     2.5 \u00d7  10 6      sensor readings that had been collected over more than nine months. The evaluation results emphasize the performance contrast under the operation of each paradigm and support a specific gap handling approach for better performance.<\/jats:p>","DOI":"10.3390\/a12100217","type":"journal-article","created":{"date-parts":[[2019,10,17]],"date-time":"2019-10-17T11:07:59Z","timestamp":1571310479000},"page":"217","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Can People Really Do Nothing? Handling Annotation Gaps in ADL Sensor Data"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0003-6914","authenticated-orcid":false,"given":"Alaa E.","family":"Abdel Hakim","sequence":"first","affiliation":[{"name":"Department of Computer Science in Jamoum, Umm Al-Qura University, Makkah 25371, Saudi Arabia"},{"name":"Electrical Engineering Department, Assiut University, Assiut 71516, Egypt"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1814-2643","authenticated-orcid":false,"given":"Wael","family":"Deabes","sequence":"additional","affiliation":[{"name":"Department of Computer Science in Jamoum, Umm Al-Qura University, Makkah 25371, Saudi Arabia"},{"name":"Department of Computers and Systems Engineering, Mansoura University, Mansoura 35516, Egypt"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,10,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1109\/TPAMI.2014.2299812","article-title":"Towards making unlabeled data never hurt","volume":"37","author":"Li","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_2","unstructured":"Weber, N. (2017). Unsupervised Learning in Human Activity Recognition: A First Foray into Clustering Data Gathered from Wearable Sensors. [Ph.D. Thesis, Radboud University]."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"829","DOI":"10.1109\/TASE.2013.2256349","article-title":"An unsupervised approach for automatic activity recognition based on Hidden Markov Model regression","volume":"10","author":"Trabelsi","year":"2013","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Toda, T., Inoue, S., Tanaka, S., and Ueda, N. (2014, January 13\u201317). Training Human Activity Recognition for Labels with Inaccurate Time Stamps. Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, Seattle, WA, USA.","DOI":"10.1145\/2638728.2641297"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Yoon, J.W., and Cho, S.B. (2011, January 23\u201325). Global\/Local Hybrid Learning of Mixture-of-Experts from Labeled and Unlabeled Data. Proceedings of the International Conference on Hybrid Artificial Intelligence Systems, Wroclaw, Poland.","DOI":"10.1007\/978-3-642-21219-2_57"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"192","DOI":"10.1016\/j.neucom.2017.04.033","article-title":"Semi-supervised multi-label classification using incomplete label information","volume":"260","author":"Tan","year":"2017","journal-title":"Neurocomputing"},{"key":"ref_7","unstructured":"Dara, R., Kremer, S., and Stacey, D. (2002, January 12\u201317). Clustering unlabeled data with SOMs improves classification of labeled real-world data. Proceedings of the 2002 International Joint Conference on Neural Networks, Honolulu, HI, USA."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1016\/j.neucom.2013.05.044","article-title":"Activity recognition with android phone using mixture-of-experts co-trained with labeled and unlabeled data","volume":"126","author":"Lee","year":"2014","journal-title":"Neurocomputing"},{"key":"ref_9","unstructured":"Pentina, A., and Lampert, C.H. (2017, January 6\u201311). Multi-Task Learning with Labeled and Unlabeled Tasks. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1613\/jair.1509","article-title":"Learning from labeled and unlabeled data: An empirical study across techniques and domains","volume":"23","author":"Chawla","year":"2005","journal-title":"J. Artif. Intell. Res."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"769","DOI":"10.1109\/72.846747","article-title":"General fuzzy min-max neural network for clustering and classification","volume":"11","author":"Gabrys","year":"2000","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_12","first-page":"17","article-title":"Semi-supervised Clustering with User Feedback","volume":"4","author":"Cohn","year":"2003","journal-title":"Constrained Clust. Adv. Algorithms Theory Appl."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Kowsari, K., Brown, D.E., Heidarysafa, M., Meimandi, K.J., Gerber, M.S., and Barnes, L.E. (2017, January 18\u201321). HDLTex: Hierarchical Deep Learning for Text Classification. Proceedings of the 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA), Cancun, Mexico.","DOI":"10.1109\/ICMLA.2017.0-134"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"28","DOI":"10.3389\/frobt.2015.00028","article-title":"A Review of Human Activity Recognition Methods","volume":"2","author":"Vrigkas","year":"2015","journal-title":"Front. Robot. AI"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2332","DOI":"10.1016\/j.neucom.2010.02.023","article-title":"A modified Markov clustering approach to unsupervised classification of protein sequences","volume":"73","year":"2010","journal-title":"Neurocomputing"},{"key":"ref_16","unstructured":"Goutte, C., D\u00e9jean, H., Gaussier, E., Cancedda, N., and Renders, J.M. (September, January 31). Combining labelled and unlabelled data: A case study on fisher kernels and transductive inference for biological entity recognition. Proceedings of the 6th Conference on Natural Language Learning, Taipei, Taiwan."},{"key":"ref_17","unstructured":"Goldman, S.A., and Zhou, Y. (July, January 29). Enhancing Supervised Learning with Unlabeled Data. Proceedings of the 17th International Conference on Machine Learning, Stanford, CA, USA."},{"key":"ref_18","unstructured":"Sammut, C., and Webb, G.I. (2001). Learning with labeled and unlabeled data. Encyclopedia of Machine Learning, Springer."},{"key":"ref_19","unstructured":"Cozman, F.G., Brazil, S.P., and Cohen, I. (2002, January 14\u201316). Unlabeled Data Can Degrade Classification Performance of Generative Classifiers. Proceedings of the Fifteenth International Florida Artificial Intelligence Research Society Conference, Pensacola, FL, USA."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s10844-006-0865-3","article-title":"Incorporating large unlabeled data to enhance em classification","volume":"26","author":"Wu","year":"2006","journal-title":"J. Intell. Inf. Syst."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1016\/j.scs.2016.07.001","article-title":"Hidden Markov Models for indirect classification of occupant behaviour","volume":"27","author":"Liisberg","year":"2016","journal-title":"Sustain. Cities Soc."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Oliver, N., and Horvitz, E. (2005, January 24\u201329). A Comparison of HMMs and Dynamic Bayesian Networks for Recognizing Office Activities. Proceedings of the International Conference on User Modeling, Edinburgh, UK.","DOI":"10.1007\/11527886_26"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Banerjee, T., Keller, J.M., and Skubie, M. (2014, January 6\u201311). Building a framework for recognition of activities of daily living from depth images using fuzzy logic. Proceedings of the 2014 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Beijing, China.","DOI":"10.1109\/FUZZ-IEEE.2014.6891647"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"11953","DOI":"10.3390\/s150511953","article-title":"User Activity Recognition in Smart Homes Using Pattern Clustering Applied to Temporal ANN Algorithm","volume":"15","author":"Bourobou","year":"2015","journal-title":"Sensors"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1109\/5.18626","article-title":"A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition","volume":"77","author":"Rabiner","year":"1989","journal-title":"Proc. IEEE"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"16:1","DOI":"10.1145\/1922649.1922653","article-title":"Human activity analysis: A review","volume":"43","author":"Aggarwal","year":"2011","journal-title":"ACM Comput. Surv."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Abdel-Hakim, A.E., and Deabes, W.A. (2017, January 12\u201315). Impact of sensor data glut on activity recognition in smart environments. Proceedings of the 2017 IEEE 17th International Conference on Ubiquitous Wireless Broadband (ICUWB), Salamanca, Spain.","DOI":"10.1109\/ICUWB.2017.8250976"},{"key":"ref_28","unstructured":"(2017, March 20). Datasets from WSU CASAS Smart Home Project. Available online: http:\/\/ailab.wsu.edu\/casas\/datasets\/."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/12\/10\/217\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:27:16Z","timestamp":1760189236000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/12\/10\/217"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,10,17]]},"references-count":28,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2019,10]]}},"alternative-id":["a12100217"],"URL":"https:\/\/doi.org\/10.3390\/a12100217","relation":{},"ISSN":["1999-4893"],"issn-type":[{"type":"electronic","value":"1999-4893"}],"subject":[],"published":{"date-parts":[[2019,10,17]]}}}