{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T08:22:42Z","timestamp":1774513362417,"version":"3.50.1"},"reference-count":31,"publisher":"MDPI AG","issue":"20","license":[{"start":{"date-parts":[[2019,10,18]],"date-time":"2019-10-18T00:00:00Z","timestamp":1571356800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100010843","name":"Guangzhou Science, Technology and Innovation Commission","doi-asserted-by":"publisher","award":["EF003\/FST-FSJ\/2019\/GSTIC"],"award-info":[{"award-number":["EF003\/FST-FSJ\/2019\/GSTIC"]}],"id":[{"id":"10.13039\/501100010843","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010843","name":"Guangzhou Science, Technology and Innovation Commission","doi-asserted-by":"publisher","award":["EF004\/FST-FSJ\/2019\/GSTIC"],"award-info":[{"award-number":["EF004\/FST-FSJ\/2019\/GSTIC"]}],"id":[{"id":"10.13039\/501100010843","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The Internet of Things (IoT) and sensors are becoming increasingly popular, especially in monitoring large and ambient environments. Applications that embrace IoT and sensors often require mining the data feeds that are collected at frequent intervals for intelligence. Despite the fact that such sensor data are massive, most of the data contents are identical and repetitive; for example, human traffic in a park at night. Most of the traditional classification algorithms were originally formulated decades ago, and they were not designed to handle such sensor data effectively. Hence, the performance of the learned model is often poor because of the small granularity in classification and the sporadic patterns in the data. To improve the quality of data mining from the IoT data, a new pre-processing methodology based on subspace similarity detection is proposed. Our method can be well integrated with traditional data mining algorithms and anomaly detection methods. The pre-processing method is flexible for handling similar kinds of sensor data that are sporadic in nature that exist in many ambient sensing applications. The proposed methodology is evaluated by extensive experiment with a collection of classical data mining models. An improvement over the precision rate is shown by using the proposed method.<\/jats:p>","DOI":"10.3390\/s19204536","type":"journal-article","created":{"date-parts":[[2019,10,18]],"date-time":"2019-10-18T11:24:15Z","timestamp":1571397855000},"page":"4536","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["A Novel Sensor Data Pre-Processing Methodology for the Internet of Things Using Anomaly Detection and Transfer-By-Subspace-Similarity Transformation"],"prefix":"10.3390","volume":"19","author":[{"given":"Yan","family":"Zhong","sequence":"first","affiliation":[{"name":"Department of Big Data and Cloud Computing, Zhuhai Institutes of Advanced Technology of the Chinese Academy of Sciences, Zhuhai 519000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Simon","family":"Fong","sequence":"additional","affiliation":[{"name":"Department of Computer and Information Science, University of Macau, Taipa 999078, Macau"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shimin","family":"Hu","sequence":"additional","affiliation":[{"name":"Department of Computer and Information Science, University of Macau, Taipa 999078, Macau"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Raymond","family":"Wong","sequence":"additional","affiliation":[{"name":"School of Computer Science &amp; Engineering, University of New South Wales, Sydney 2052, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Weiwei","family":"Lin","sequence":"additional","affiliation":[{"name":"School of Computer Science and Engineering, South China University of Technology, Guangzhou 510006, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,10,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Dhakar, L., Tay, F., and Lee, C. (2015, January 18\u201322). Skin based flexible triboelectric nanogenerators with motion sensing capability. Proceedings of the 2015 28th IEEE International Conference on Micro Electro Mechanical Systems (MEMS), Estoril, Portugal.","DOI":"10.1109\/MEMSYS.2015.7050897"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Li, T., Liu, Y., Tian, Y., Shen, S., and Mao, W. (2012, January 20\u201323). A storage solution for massive iot data based on nosql. Proceedings of the 2012 IEEE International Conference on Green Computing and Communications, Besancon, France.","DOI":"10.1109\/GreenCom.2012.18"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"164","DOI":"10.4236\/jcc.2015.35021","article-title":"Internet of Things (IoT): A literature review","volume":"3","author":"Madakam","year":"2015","journal-title":"J. Comput. Commun."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"3475","DOI":"10.1016\/j.jbiomech.2008.08.009","article-title":"A wearable system for pre-impact fall detection","volume":"41","author":"Nyan","year":"2008","journal-title":"J. Biomech."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Dan, L., Xin, C., Chongwei, H., and Liangliang, J. (2015, January 19\u201320). Intelligent agriculture greenhouse environment monitoring system based on IOT technology. Proceedings of the 2015 International Conference on Intelligent Transportation, Big Data and Smart City, Halong Bay, Vietnam.","DOI":"10.1109\/ICITBS.2015.126"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Shi, F., Li, Q., Zhu, T., and Ning, H. (2018). A survey of data semantization in internet of things. Sensors, 18.","DOI":"10.3390\/s18010313"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Xie, S., and Chen, Z. (2017). Anomaly detection and redundancy elimination of big sensor data in internet of things. arXiv.","DOI":"10.1007\/978-3-319-42553-5_16"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"P\u00e9rez-Penichet, C., Hermans, F., Varshney, A., and Voigt, T. (2016, January 3\u20137). Augmenting IoT networks with backscatter-enabled passive sensor tags. Proceedings of the 3rd Workshop on Hot Topics in Wireless, New York, NY, USA.","DOI":"10.1145\/2980115.2980132"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Nesa, N., Ghosh, T., and Banerjee, I. (2018, January 15\u201318). Outlier detection in sensed data using statistical learning models for IoT. Proceedings of the 2018 IEEE Wireless Communications and Networking Conference (WCNC), Barcelona, Spain.","DOI":"10.1109\/WCNC.2018.8376988"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"20150202","DOI":"10.1098\/rsta.2015.0202","article-title":"Principal component analysis: A review and recent developments","volume":"374","author":"Jolliffe","year":"2016","journal-title":"Philos. Trans. R. Soc. A: Math. Phys. Eng. Sci."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Patro, S., and Sahu, K.K. (2015). Normalization: A preprocessing stage. arXiv.","DOI":"10.17148\/IARJSET.2015.2305"},{"key":"ref_12","unstructured":"M&M Research Group (2012). Internet of Things (IoT) & M2M Communication Market: Advanced Technologies, Future Cities & Adoption Trends, Roadmaps & Worldwide Forecasts 2012\u20132017, Electronics. ca Publications. Technical Report."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2787","DOI":"10.1016\/j.comnet.2010.05.010","article-title":"The internet of things: A survey","volume":"54","author":"Atzori","year":"2010","journal-title":"Comput. Netw."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1497","DOI":"10.1016\/j.adhoc.2012.02.016","article-title":"Internet of things: Vision, applications and research challenges","volume":"10","author":"Miorandi","year":"2012","journal-title":"Ad Hoc Netw."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1007\/s11277-011-0288-5","article-title":"Internet of things: Applications and challenges in technology and standardization","volume":"58","author":"Bandyopadhyay","year":"2011","journal-title":"Wirel. Pers. Commun."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Xu, L., Guo, D., Tay, F.E.H., and Xing, S. (2010, January 20\u201321). A wearable vital signs monitoring system for pervasive healthcare. Proceedings of the 2010 IEEE Conference on Sustainable Utilization and Development in Engineering and Technology, Petaling Jaya, Malaysia.","DOI":"10.1109\/STUDENT.2010.5687003"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Cantoni, V., Lombardi, L., and Lombardi, P. (2006, January 20\u201324). Challenges for data mining in distributed sensor networks. Proceedings of the 18th International Conference on Pattern Recognition, Hong Kong, China.","DOI":"10.1109\/ICPR.2006.359"},{"key":"ref_18","unstructured":"Keller, T. (2011). Mining the Internet of Things-Detection of False-Positive RFID Tag Reads Using Low-Level Reader Data. [Ph.D. Thesis, University of St. Gallen]."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Masciari, E. (2007, January 6\u20138). A Framework for Outlier Mining in RFID data. Proceedings of the 11th International Database Engineering and Applications Symposium, Banff, AB, Canada.","DOI":"10.1109\/IDEAS.2007.4318112"},{"key":"ref_20","unstructured":"Bin, S., Yuan, L., and Xiaoyi, W. (2010, January 9\u201311). Research on data mining models for the internet of things. Proceedings of the 2010 International Conference on Image Analysis and Signal Processing (IASP), Zhejiang, China."},{"key":"ref_21","unstructured":"MacQueen, J. (July, January 21). Some methods for classification and analysis of multivariate observations. Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Oakland, CA, USA."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"264","DOI":"10.1145\/331499.331504","article-title":"Data clustering: A review","volume":"31","author":"Jain","year":"1999","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"645","DOI":"10.1109\/TNN.2005.845141","article-title":"Survey of clustering algorithms","volume":"16","author":"Xu","year":"2005","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"660","DOI":"10.1109\/21.97458","article-title":"A survey of decision tree classifier methodology","volume":"21","author":"Safavian","year":"1991","journal-title":"IEEE Trans. Syst. Man, Cybern."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"399","DOI":"10.1016\/S0034-4257(97)00049-7","article-title":"Decision tree classification of land cover from remotely sensed data","volume":"61","author":"Friedl","year":"1997","journal-title":"Remote Sens. Environ."},{"key":"ref_26","unstructured":"McCallum, A., and Nigam, K. (1998, January 26\u201327). A comparison of event models for naive bayes text classification. Proceedings of the AAAI-98 Workshop on Learning for Text Categorization Citeseer, Madison, WI, USA."},{"key":"ref_27","first-page":"223","article-title":"An analysis of Bayesian classifiers","volume":"90","author":"Langley","year":"1992","journal-title":"Aaai"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/5254.708428","article-title":"Support vector machines","volume":"13","author":"Hearst","year":"1998","journal-title":"IEEE Intell. Syst. Appl."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Danita, M., Mathew, B., Shereen, N., Sharon, N., and Paul, J.J. (2018, January 14\u201315). IoT Based Automated Greenhouse Monitoring System. Proceedings of the 2018 Second International Conference on Intelligent Computing and Control Systems (ICICCS), Madurai, India.","DOI":"10.1109\/ICCONS.2018.8662911"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1571","DOI":"10.1109\/JIOT.2017.2712672","article-title":"Predictive analytics for complex IoT data streams","volume":"4","author":"Akbar","year":"2017","journal-title":"IEEE Int. Things J."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Liu, F.T., Ting, K.M., and Zhou, Z.H. (2008, January 15\u201319). Isolation forest. Proceedings of the Eighth IEEE International Conference on Data Mining, Pisa, Italy.","DOI":"10.1109\/ICDM.2008.17"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/20\/4536\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:27:39Z","timestamp":1760189259000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/20\/4536"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,10,18]]},"references-count":31,"journal-issue":{"issue":"20","published-online":{"date-parts":[[2019,10]]}},"alternative-id":["s19204536"],"URL":"https:\/\/doi.org\/10.3390\/s19204536","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,10,18]]}}}