{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,27]],"date-time":"2026-06-27T05:29:39Z","timestamp":1782538179158,"version":"3.54.5"},"reference-count":31,"publisher":"MDPI AG","issue":"22","license":[{"start":{"date-parts":[[2020,11,13]],"date-time":"2020-11-13T00:00:00Z","timestamp":1605225600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper evaluates data stream classifiers from the perspective of connected devices, focusing on the use case of Human Activity Recognition. We measure both the classification performance and resource consumption (runtime, memory, and power) of five usual stream classification algorithms, implemented in a consistent library, and applied to two real human activity datasets and three synthetic datasets. Regarding classification performance, the results show the overall superiority of the Hoeffding Tree, the Mondrian forest, and the Na\u00efve Bayes classifiers over the Feedforward Neural Network and the Micro Cluster Nearest Neighbor classifiers on four datasets out of six, including the real ones. In addition, the Hoeffding Tree and\u2014to some extent\u2014the Micro Cluster Nearest Neighbor, are the only classifiers that can recover from a concept drift. Overall, the three leading classifiers still perform substantially worse than an offline classifier on the real datasets. Regarding resource consumption, the Hoeffding Tree and the Mondrian forest are the most memory intensive and have the longest runtime; however, no difference in power consumption is found between classifiers. We conclude that stream learning for Human Activity Recognition on connected objects is challenged by two factors which could lead to interesting future work: a high memory consumption and low F1 scores overall.<\/jats:p>","DOI":"10.3390\/s20226486","type":"journal-article","created":{"date-parts":[[2020,11,13]],"date-time":"2020-11-13T08:44:02Z","timestamp":1605257042000},"page":"6486","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["A Benchmark of Data Stream Classification for Human Activity Recognition on Connected Objects"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2129-5517","authenticated-orcid":false,"given":"Martin","family":"Khannouz","sequence":"first","affiliation":[{"name":"Department of Computer Science and Software Engineering, Concordia University, Montr\u00e9al, QC H3G 1M8, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2620-5883","authenticated-orcid":false,"given":"Tristan","family":"Glatard","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Software Engineering, Concordia University, Montr\u00e9al, QC H3G 1M8, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,11,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1016\/S1389-1286(01)00302-4","article-title":"Wireless Sensor Networks: A Survey","volume":"38","author":"Akyildiz","year":"2002","journal-title":"Comput. Netw."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Duarte-Melo, E., and Liu, M. (2002, January 17\u201321). Analysis of energy consumption and lifetime of heterogeneous wireless sensor networks. Proceedings of the Global Telecommunications Conference. GLOBECOM 02, Taipei, Taiwan.","DOI":"10.1109\/GLOCOM.2002.1188034"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2040","DOI":"10.14778\/2824032.2824132","article-title":"Real time analytics: Algorithms and systems","volume":"8","author":"Kejariwal","year":"2015","journal-title":"Proc. VLDB Endow."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Gama, J.A., Sebasti\u00e3o, R., and Rodrigues, P.P. (2009, January 28). Issues in Evaluation of Stream Learning Algorithms. Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Paris, France.","DOI":"10.1145\/1557019.1557060"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Bifet, A., Zhang, J., Fan, W., He, C., Zhang, J., Qian, J., Holmes, G., and Pfahringer, B. (2017, January 13). Extremely Fast Decision Tree Mining for Evolving Data Streams. Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining-KDD \u201917, Halifax, NS, Canada.","DOI":"10.1145\/3097983.3098139"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"201","DOI":"10.14257\/ijdta.2016.9.9.19","article-title":"Stream Data Mining: Platforms, Algorithms, Performance Evaluators and Research Trends","volume":"9","author":"Prasad","year":"2016","journal-title":"Int. J. Database Theory Appl."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Kaur, A., and Kumar, R. (2020). A Comprehensive Analysis of Classification Methods for Big Data Stream, Springer.","DOI":"10.1007\/978-981-15-0222-4_18"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Priya, S., and Uthra, R.A. (2020). Comprehensive analysis for class imbalance data with concept drift using ensemble based classification. J. Ambient Intell. Humaniz. Comput.","DOI":"10.1007\/s12652-020-01934-y"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1013","DOI":"10.1109\/TNSM.2020.2983921","article-title":"Evaluating the Four-way Performance Trade-off for Data Stream Classification in Edge Computing","volume":"17","author":"Fernandes","year":"2020","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_10","first-page":"165","article-title":"Human Activity Recognition from Wireless Sensor Network Data: Benchmark and Software","volume":"4","author":"Englebienne","year":"2011","journal-title":"Act. Recognit. Pervasive Intell. Environ."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Amato, G., Bacciu, D., Chessa, S., Dragone, M., Gallicchio, C., Gennaro, C., Lozano, H., Micheli, A., O\u2019Hare, G., and Renteria, A. (2016). A Benchmark Dataset for Human Activity Recognition and Ambient Assisted Living. International Symposium on Ambient Intelligence, Springer.","DOI":"10.1007\/978-3-319-40114-0_1"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2033","DOI":"10.1016\/j.patrec.2012.12.014","article-title":"The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition","volume":"23","author":"Chavarriaga","year":"2013","journal-title":"Pattern Recognit. Lett."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Janidarmian, M., Fekr, A.R., Radecka, K., and Zilic, Z. (2017). A Comprehensive Analysis on Wearable Acceleration Sensors in Human Activity Recognition. Sensors, 17.","DOI":"10.3390\/s17030529"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1080\/00949655.2014.925192","article-title":"Runtime and memory consumption analyses for machine learning R programs","volume":"85","author":"Kotthaus","year":"2015","journal-title":"J. Stat. Comput. Simul"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Ugulino, W., Cardador, D., Vega, K., Velloso, E., Milidi\u00fa, R., and Fuks, H. (2012). Wearable Computing: Accelerometers\u2019 Data Classification of Body Postures and Movements. Advances in Artificial Intelligence-SBIA 2012, Springer.","DOI":"10.1007\/978-3-642-34459-6_6"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Ahmed, N., Kabir, R., Rahman, A., Momin, A., and Islam, M.R. (2019, January 3\u20136). Smartphone Sensor Based Physical Activity Identification by Using Hardware-Efficient Support Vector Machines for Multiclass Classification. Proceedings of the 2019 IEEE Eurasia Conference on IOT, Communication and Engineering (ECICE), Yunlin, Taiwan.","DOI":"10.1109\/ECICE47484.2019.8942781"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1016\/j.engappai.2018.04.002","article-title":"Robust Human Activity Recognition Using Smartwatches and Smartphones","volume":"72","author":"Blunck","year":"2018","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Sarbishei, O. (2019, January 15\u201318). A Platform and Methodology Enabling Real-Time Motion Pattern Recognition on Low-Power Smart Devices. Proceedings of the 2019 IEEE 5th World Forum on Internet of Things (WF-IoT), Limerick, Ireland.","DOI":"10.1109\/WF-IoT.2019.8767219"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1485","DOI":"10.21105\/joss.01485","article-title":"OrpailleCC: A Library for Data Stream Analysis on Embedded Systems","volume":"4","author":"Khannouz","year":"2019","journal-title":"J. Open Source Softw."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Bifet, A., Maniu, S., Qian, J., Tian, G., He, C., and Fan, W. (2015, January 14\u201317). StreamDM: Advanced Data Mining in Spark Streaming. Proceedings of the 2015 IEEE International Conference on Data Mining Workshop ICDMW\u201915, Atlantic City, NJ, USA.","DOI":"10.1109\/ICDMW.2015.140"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Fan, B., Andersen, D.G., Kaminsky, M., and Mitzenmacher, M.D. (2014). Cuckoo Filter: Practically Better Than Bloom. Proceedings of the 10th ACM International on Conference on Emerging Networking Experiments and Technologies, Sydney, Australia, 2 December 2014, Association for Computing Machinery.","DOI":"10.1145\/2674005.2674994"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Li, B., Sarbishei, O., Nourani, H., and Glatard, T. (2018, January 10\u201313). A multi-dimensional extension of the Lightweight Temporal Compression method. Proceedings of the 2018 IEEE International Conference on Big Data, Seattle, WA, USA.","DOI":"10.1109\/BigData.2018.8621946"},{"key":"ref_23","first-page":"3140","article-title":"Mondrian Forests: Efficient Online Random Forests","volume":"Volume 4","author":"Ghahramani","year":"2014","journal-title":"Advances in Neural Information Processing Systems 27"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1145\/3373464.3373470","article-title":"Machine Learning for Streaming Data: State of the Art, Challenges, and Opportunities","volume":"21","author":"Gomes","year":"2019","journal-title":"SIGKDD Explor. Newsl."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1016\/j.future.2017.03.026","article-title":"Scalable real-time classification of data streams with concept drift","volume":"75","author":"Tennant","year":"2017","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Lewis, D.D. (1998). Naive (Bayes) at Forty: The Independence Assumption in Information Retrieval. Proceedings of the 10th European Conference on Machine Learning, Springer.","DOI":"10.1007\/BFb0026666"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Domingos, P., and Hulten, G. (2000, January 1). Mining High-Speed Data Streams. Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Boston, MA, USA.","DOI":"10.1145\/347090.347107"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"6474","DOI":"10.3390\/s140406474","article-title":"Window Size Impact in Human Activity Recognition","volume":"14","author":"Banos","year":"2014","journal-title":"Sensors"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Morris, D., Saponas, T.S., Guillory, A., and Kelner, I. (2014, January 6). RecoFit: Using a Wearable Sensor to Find, Recognize, and Count Repetitive Exercises. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Toronto, ON, Canada.","DOI":"10.1145\/2556288.2557116"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Dehghani, A., Sarbishei, O., Glatard, T., and Shihab, E. (2019). A Quantitative Comparison of Overlapping and Non-Overlapping Sliding Windows for Human Activity Recognition Using Inertial Sensors. Sensors, 19.","DOI":"10.3390\/s19225026"},{"key":"ref_31","first-page":"1601","article-title":"MOA: Massive Online Analysis","volume":"11","author":"Bifet","year":"2010","journal-title":"J. Mach. Learn. Res."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/22\/6486\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:33:04Z","timestamp":1760178784000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/22\/6486"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,13]]},"references-count":31,"journal-issue":{"issue":"22","published-online":{"date-parts":[[2020,11]]}},"alternative-id":["s20226486"],"URL":"https:\/\/doi.org\/10.3390\/s20226486","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,11,13]]}}}