{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T11:26:34Z","timestamp":1780053994099,"version":"3.54.0"},"reference-count":46,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2018,7,3]],"date-time":"2018-07-03T00:00:00Z","timestamp":1530576000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Automatic data annotation eliminates most of the challenges we faced due to the manual methods of annotating sensor data. It significantly improves users\u2019 experience during sensing activities since their active involvement in the labeling process is reduced. An unsupervised learning technique such as clustering can be used to automatically annotate sensor data. However, the lingering issue with clustering is the validation of generated clusters. In this paper, we adopted the k-means clustering algorithm for annotating unlabeled sensor data for the purpose of detecting sensitive location information of mobile crowd sensing users. Furthermore, we proposed a cluster validation index for the k-means algorithm, which is based on Multiple Pair-Frequency. Thereafter, we trained three classifiers (Support Vector Machine, K-Nearest Neighbor, and Na\u00efve Bayes) using cluster labels generated from the k-means clustering algorithm. The accuracy, precision, and recall of these classifiers were evaluated during the classification of \u201cnon-sensitive\u201d and \u201csensitive\u201d data from motion and location sensors. Very high accuracy scores were recorded from Support Vector Machine and K-Nearest Neighbor classifiers while a fairly high accuracy score was recorded from the Na\u00efve Bayes classifier. With the hybridized machine learning (unsupervised and supervised) technique presented in this paper, unlabeled sensor data was automatically annotated and then classified.<\/jats:p>","DOI":"10.3390\/s18072134","type":"journal-article","created":{"date-parts":[[2018,7,3]],"date-time":"2018-07-03T11:12:58Z","timestamp":1530616378000},"page":"2134","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Automatic Annotation of Unlabeled Data from Smartphone-Based Motion and Location Sensors"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4840-9345","authenticated-orcid":false,"given":"Nsikak","family":"Pius Owoh","sequence":"first","affiliation":[{"name":"School of Computer Sciences, Universiti Sains Malaysia, 11800 USM Penang, Malaysia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8081-5223","authenticated-orcid":false,"given":"Manmeet","family":"Mahinderjit Singh","sequence":"additional","affiliation":[{"name":"School of Computer Sciences, Universiti Sains Malaysia, 11800 USM Penang, Malaysia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zarul Fitri","family":"Zaaba","sequence":"additional","affiliation":[{"name":"School of Computer Sciences, Universiti Sains Malaysia, 11800 USM Penang, Malaysia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2018,7,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1109\/TSUSC.2017.2666043","article-title":"A cost-effective distributed framework for data collection in cloud-based mobile crowd sensing architectures","volume":"2","author":"Capponi","year":"2017","journal-title":"IEEE Trans. Sustain. Comput."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1109\/MCOM.2011.6069707","article-title":"Mobile crowdsensing: Current state and future challenges","volume":"49","author":"Ganti","year":"2011","journal-title":"IEEE Commun. Mag."},{"key":"ref_3","first-page":"32","article-title":"Fostering ParticipAction in Smart Cities: A Geo-Social Crowdsensing Platform","volume":"48","author":"Cardone","year":"2013","journal-title":"IEEE Commun. Mag."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"402","DOI":"10.1109\/SURV.2012.031412.00077","article-title":"Mobile Phone Sensing Systems: A Survey","volume":"15","author":"Khan","year":"2013","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1109\/MCOM.2014.6917406","article-title":"The participact mobile crowd sensing living lab: The testbed for smart cities","volume":"52","author":"Cardone","year":"2014","journal-title":"IEEE Commun. Mag."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1109\/JSYST.2015.2460747","article-title":"Health-CPS: Healthcare Cyber-Physical System Assisted by Cloud and Big Data","volume":"17","author":"Zhang","year":"2017","journal-title":"IEEE Syst. J."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1281","DOI":"10.1214\/14-AOAS742","article-title":"Joint modeling of multiple time series via the beta process with application to motion capture segmentation","volume":"8","author":"Fox","year":"2014","journal-title":"Ann. Appl. Stat."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1007\/s11036-013-0484-5","article-title":"Bewell: Sensing sleep, physical activities and social interactions to promote wellbeing","volume":"19","author":"Lane","year":"2014","journal-title":"Mob. Netw. Appl."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Lane, N.D., Chon, Y., Zhou, L., Zhang, Y., Li, F., Kim, D., Ding, G., Zhao, F., and Cha, H. (2013, January 11\u201315). Piggyback CrowdSensing (PCS): Energy efficient crowdsourcing of mobile sensor data by exploiting smartphone app opportunities. Proceedings of the 11th ACM Conference on Embedded Networked Sensor Systems, Rome, Italy.","DOI":"10.1145\/2517351.2517372"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Martindale, C.F., Hoenig, F., Strohrmann, C., and Eskofier, B.M. (2017). Smart Annotation of Cyclic Data Using Hierarchical Hidden Markov Models. Sensors, 17.","DOI":"10.3390\/s17102328"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"161","DOI":"10.3233\/THC-2009-0546","article-title":"Annotating smart environment sensor data for activity learning","volume":"17","author":"Szewcyzk","year":"2009","journal-title":"Technol. Health Care"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2521","DOI":"10.1109\/TPAMI.2011.36","article-title":"Weakly supervised recognition of daily life activities with wearable sensors","volume":"33","author":"Stikic","year":"2011","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Huynh, T., Fritz, M., and Schiele, B. (2008, January 21\u201324). Discovery of activity patterns using topic models. Proceedings of the 10th International Conference on Ubiquitous Computing, Seoul, Korea.","DOI":"10.1145\/1409635.1409638"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"242","DOI":"10.1016\/j.pmcj.2014.05.006","article-title":"Using unlabeled data in a sparse-coding framework for human activity recognition","volume":"15","author":"Bhattacharya","year":"2014","journal-title":"Pervas. Mob. Comput."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Maekawa, T., and Watanabe, S. (2011, January 12\u201315). Unsupervised activity recognition with user\u2019s physical characteristics data. Proceedings of the 15th Annual International Symposium on Wearable Computers (ISWC), San Francisco, CA, USA.","DOI":"10.1109\/ISWC.2011.24"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Riboni, D., Sztyler, T., Civitarese, G., and Stuckenschmidt, H. (2016, January 12\u201316). Unsupervised recognition of interleaved activities of daily living through ontological and probabilistic reasoning. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Heidelberg, Germany.","DOI":"10.1145\/2971648.2971691"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Kapadia, A., Kotz, D., and Triandopoulos, N. (2009, January 5\u201310). Opportunistic sensing: Security challenges for the new paradigm. Proceedings of the Communication Systems and Networks and Workshops, Bangalore, India.","DOI":"10.1109\/COMSNETS.2009.4808850"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3098","DOI":"10.1109\/TKDE.2016.2592527","article-title":"Scalable daily human behavioral pattern mining from multivariate temporal data","volume":"28","author":"Rawassizadeh","year":"2016","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Miluzzo, E., Lane, N.D., Fodor, K., Peterson, R., Lu, H., Musolesi, M., Eisenman, S.B., Zheng, X., and Campbell, A.T. (2008, January 4\u20137). Sensing meets mobile social networks: The design, implementation and evaluation of the cenceme application. Proceedings of the 6th ACM Conference on Embedded Network Sensor Systems, Raleigh, NC, USA.","DOI":"10.1145\/1460412.1460445"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1145\/2499621","article-title":"A tutorial on human activity recognition using body-worn inertial sensors","volume":"46","author":"Bulling","year":"2014","journal-title":"ACM Comput. Surv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Radu, V., Lane, N.D., Bhattacharya, S., Mascolo, C., Marina, M.K., and Kawsar, F. (2016, January 12\u201316). Towards multimodal deep learning for activity recognition on mobile devices. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct, Heidelberg, Germany.","DOI":"10.1145\/2968219.2971461"},{"key":"ref_22","unstructured":"Hammerla, N.Y., Halloran, S., and Ploetz, T. (arXiv, 2016). Deep, convolutional, and recurrent models for human activity recognition using wearables, arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1016\/j.eswa.2016.04.032","article-title":"Human activity recognition with smartphone sensors using deep learning neural networks","volume":"59","author":"Ronao","year":"2016","journal-title":"Expert Syst. Appl."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Ord\u00f3\u00f1ez, F.J., and Roggen, D. (2016). Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors, 16.","DOI":"10.3390\/s16010115"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"687","DOI":"10.1109\/TBME.2011.2178070","article-title":"Multisensor data fusion for physical activity assessment","volume":"59","author":"Liu","year":"2012","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Calatroni, A., Roggen, D., and Tr\u00f6ster, G. (2011, January 9\u201312). Collection and curation of a large reference dataset for activity recognition. Proceedings of the 2011 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Anchorage, AK, USA.","DOI":"10.1109\/ICSMC.2011.6083638"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Liu, R., Chen, T., and Huang, L. (2010, January 11\u201314). Research on human activity recognition based on active learning. Proceedings of the 2010 International Conference on Machine Learning and Cybernetics (ICMLC), Qingdao, China.","DOI":"10.1109\/ICMLC.2010.5581050"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Miu, T., Missier, P., and Pl\u00f6tz, T. (2015, January 26\u201328). Bootstrapping personalised human activity recognition models using online active learning. Proceedings of the 2015 IEEE International Conference on Computer and Information Technology, Liverpool, UK.","DOI":"10.1109\/CIT\/IUCC\/DASC\/PICOM.2015.170"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Yao, L., Nie, F., Sheng, Q.Z., Gu, T., Li, X., and Wang, S. (2016, January 12\u201316). Learning from less for better: Semi-supervised activity recognition via shared structure discovery. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Heidelberg, Germany.","DOI":"10.1145\/2971648.2971701"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1109\/THMS.2015.2489688","article-title":"Toward personalized activity recognition systems with a semipopulation approach","volume":"46","author":"Hong","year":"2016","journal-title":"IEEE Trans. Hum. Mach. Syst."},{"key":"ref_31","unstructured":"Guan, X., Raich, R., and Wong, W.-K. (2016, January 20\u201322). Efficient multi-instance learning for activity recognition from time series data using an auto-regressive hidden Markov model. Proceedings of the International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Nath, S. (2012, January 25\u201329). ACE: Exploiting correlation for energy-efficient and continuous context sensing. Proceedings of the 10th International Conference on Mobile Systems, Applications, and Services, Lake District, UK.","DOI":"10.1145\/2307636.2307640"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Srinivasan, V., Moghaddam, S., Mukherji, A., Rachuri, K.K., Xu, C., and Tapia, E.M. (2014, January 13\u201317). Mobileminer: Mining your frequent patterns on your phone. Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Seattle, WA, USA.","DOI":"10.1145\/2632048.2632052"},{"key":"ref_34","unstructured":"Kiukkonen, N., Blom, J., Dousse, O., Gatica-Perez, D., and Laurila, J. (2010, January 13\u201316). Towards rich mobile phone datasets: Lausanne data collection campaign. Proceedings of the ACM International Conference on Pervasive Services (ICPS 2010), Berlin, Germany."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"3490","DOI":"10.1109\/ACCESS.2017.2671678","article-title":"Crowdsensim: A simulation platform for mobile crowdsensing in realistic urban environments","volume":"5","author":"Fiandrino","year":"2017","journal-title":"IEEE Access"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1996","DOI":"10.1109\/COMST.2014.2320099","article-title":"Machine learning in wireless sensor networks: Algorithms, strategies, and applications","volume":"16","author":"Alsheikh","year":"2014","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Palma, A.T., Bogorny, V., Kuijpers, B., and Alvares, L.O. (2008, January 16\u201320). A clustering-based approach for discovering interesting places in trajectories. Proceedings of the 2008 ACM Symposium on Applied Computing, Fortaleza, Ceara, Brazil.","DOI":"10.1145\/1363686.1363886"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Zheng, Y., Li, Q., Chen, Y., Xie, X., and Ma, W.-Y. (2008, January 21\u201324). Understanding mobility based on GPS data. Proceedings of the 10th International Conference on Ubiquitous Computing, Seoul, Korea.","DOI":"10.1145\/1409635.1409677"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"965","DOI":"10.1109\/TIFS.2017.2774505","article-title":"Mobile Phone Clustering From Speech Recordings Using Deep Representation and Spectral Clustering","volume":"13","author":"Li","year":"2018","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"2936","DOI":"10.1109\/TNNLS.2016.2608354","article-title":"Cluster validation method for determining the number of clusters in categorical sequences","volume":"28","author":"Guo","year":"2017","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1016\/j.eswa.2016.09.025","article-title":"An improved overlapping k-means clustering method for medical applications","volume":"67","author":"Khanmohammadi","year":"2017","journal-title":"Expert Syst. Appl."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"651","DOI":"10.1016\/j.patrec.2009.09.011","article-title":"Data clustering: 50 years beyond K-means","volume":"31","author":"Jain","year":"2010","journal-title":"Pattern Recognit. Lett."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1016\/j.ijar.2013.03.018","article-title":"An automatic method to determine the number of clusters using decision-theoretic rough set","volume":"55","author":"Yu","year":"2014","journal-title":"Int. J. Approx. Reason."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"24884","DOI":"10.1109\/ACCESS.2017.2743985","article-title":"A Clustering Validity Index Based on Pairing Frequency","volume":"5","author":"Cui","year":"2017","journal-title":"IEEE Access"},{"key":"ref_45","unstructured":"Freedman, R. (2017). Smartphone recorded driving sensor data. Indianapolis International Airport to Urbana, IL, University of Illinois at Urbana-Champaign."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Yan, Z., Chakraborty, D., Parent, C., Spaccapietra, S., and Aberer, K. (2011, January 21\u201324). SeMiTri: A framework for semantic annotation of heterogeneous trajectories. Proceedings of the 14th International Conference on Extending Database Technology, Uppsala, Sweden.","DOI":"10.1145\/1951365.1951398"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/7\/2134\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:11:07Z","timestamp":1760195467000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/7\/2134"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,7,3]]},"references-count":46,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2018,7]]}},"alternative-id":["s18072134"],"URL":"https:\/\/doi.org\/10.3390\/s18072134","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,7,3]]}}}