{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T16:58:46Z","timestamp":1776445126064,"version":"3.51.2"},"reference-count":57,"publisher":"MDPI AG","issue":"20","license":[{"start":{"date-parts":[[2021,10,18]],"date-time":"2021-10-18T00:00:00Z","timestamp":1634515200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The recent explosive growth in the number of smart technologies relying on data collected from sensors and processed with machine learning classifiers made the training data imbalance problem more visible than ever before. Class-imbalanced sets used to train models of various events of interest are among the main reasons for a smart technology to work incorrectly or even to completely fail. This paper presents an attempt to resolve the imbalance problem in sensor sequential (time-series) data through training data augmentation. An Unrolled Generative Adversarial Networks (Unrolled GAN)-powered framework is developed and successfully used to balance the training data of smartphone accelerometer and gyroscope sensors in different contexts of road surface monitoring. Experiments with other sensor data from an open data collection are also conducted. It is demonstrated that the proposed approach allows for improving the classification performance in the case of heavily imbalanced data (the F1 score increased from 0.69 to 0.72, p&lt;0.01, in the presented case study). However, the effect is negligible in the case of slightly imbalanced or inadequate training sets. The latter determines the limitations of this study that would be resolved in future work aimed at incorporating mechanisms for assessing the training data quality into the proposed framework and improving its computational efficiency.<\/jats:p>","DOI":"10.3390\/s21206892","type":"journal-article","created":{"date-parts":[[2021,10,20]],"date-time":"2021-10-20T21:31:26Z","timestamp":1634765486000},"page":"6892","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["A Machine Learning Framework for Balancing Training Sets of Sensor Sequential Data Streams"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2158-145X","authenticated-orcid":false,"given":"Budi Darma","family":"Setiawan","sequence":"first","affiliation":[{"name":"Graduate School of Information Science and Engineering, Ritsumeikan University, Kusatsu 525-8577, Japan"},{"name":"Faculty of Computer Science, Brawijaya University, Malang 65145, Indonesia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2383-3158","authenticated-orcid":false,"given":"Uwe","family":"Serd\u00fclt","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Ritsumeikan University, Kusatsu 525-8577, Japan"},{"name":"Center for Democracy Studies Aarau (ZDA), University of Zurich, 8001 Zurich, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Victor","family":"Kryssanov","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Ritsumeikan University, Kusatsu 525-8577, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,10,18]]},"reference":[{"key":"ref_1","first-page":"1","article-title":"Industry 4.0: A Survey on Technologies, Applications and Open Research Issues","volume":"6","author":"Lu","year":"2017","journal-title":"J. Ind. Inf. Integr."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Medina, C.A., Perez, M.R., and Trujillo, L.C. (2017, January 21\u201323). IoT Paradigm into the Smart City Vision: A Survey. Proceedings of the 2017 IEEE International Conference on Internet of Things (iThings), IEEE Green Computing and Communications (GreenCom), IEEE Cyber, Physical and Social Computing (CPSCom) IEEE Smart Data (SmartData), Exeter, UK.","DOI":"10.1109\/iThings-GreenCom-CPSCom-SmartData.2017.109"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"e4233","DOI":"10.1002\/ett.4233","article-title":"6G: A Comprehensive Survey on Technologies, Applications, Challenges, and Research Problems","volume":"32","author":"Mahmoud","year":"2021","journal-title":"Trans. Emerg. Telecommun. Technol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"210816","DOI":"10.1109\/ACCESS.2020.3037715","article-title":"Human Activity Recognition Using Inertial, Physiological and Environmental Sensors: A Comprehensive Survey","volume":"8","author":"Demrozi","year":"2020","journal-title":"IEEE Access"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1016\/j.dcan.2017.10.002","article-title":"Machine Learning for Internet of Things Data Analysis: A Survey","volume":"4","author":"Mahdavinejad","year":"2018","journal-title":"Digit. Commun. Netw."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2689","DOI":"10.1016\/j.apt.2020.04.034","article-title":"Acoustic Signal Based Fault Detection on Belt Conveyor Idlers Using Machine Learning","volume":"31","author":"Liu","year":"2020","journal-title":"Adv. Powder Technol."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Serhani, M.A., El Kassabi, H.T., Ismail, H., and Navaz, A.N. (2020). ECG Monitoring Systems: Review, Architecture, Processes, and Key Challenges. Sensors, 20.","DOI":"10.3390\/s20061796"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Santos, G.L., Endo, P.T., Monteiro, K.H.D.C., Rocha, E.D.S., Silva, I., and Lynn, T. (2019). Accelerometer-Based Human Fall Detection Using Convolutional Neural Networks. Sensors, 19.","DOI":"10.3390\/s19071644"},{"key":"ref_9","first-page":"1","article-title":"A Systematic Review on Imbalanced Data Challenges in Machine Learning: Applications and Solutions","volume":"52","author":"Kaur","year":"2019","journal-title":"ACM Comput. Surv."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1186\/s13173-019-0085-7","article-title":"An IoT Sensor and Scenario Survey for Data Researchers","volume":"25","author":"Sadok","year":"2019","journal-title":"J. Brazilian Comput. Soc."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1016\/j.patcog.2019.02.023","article-title":"The Impact of Class Imbalance in Classification Performance Metrics Based on the Binary Confusion Matrix","volume":"91","author":"Luque","year":"2019","journal-title":"Pattern Recognit."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Spelmen, V.S., and Porkodi, R. (2018, January 1\u20133). A Review on Handling Imbalanced Data. Proceedings of the 2018 International Conference on Current Trends towards Converging Technologies (ICCTCT 2018), Coimbatore, India.","DOI":"10.1109\/ICCTCT.2018.8551020"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40537-021-00492-0","article-title":"Text Data Augmentation for Deep Learning","volume":"8","author":"Shorten","year":"2021","journal-title":"J. Big Data"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1016\/j.patcog.2016.03.012","article-title":"Analyzing the Oversampling of Different Classes and Types of Examples in Multi-Class Imbalanced Datasets","volume":"57","author":"Krawczyk","year":"2016","journal-title":"Pattern Recognit."},{"key":"ref_15","first-page":"104","article-title":"A KNN Undersampling Approach for Data Balancing","volume":"7","author":"Beckmann","year":"2015","journal-title":"J. Intell. Learn. Syst. Appl."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Liang, G. (2013). An Effective Method for Imbalanced Time Series Classification: Hybrid Sampling. AI 2013: Advances in Artificial Intelligence, Springer.","DOI":"10.1007\/978-3-319-03680-9_38"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Hasanin, T., and Khoshgoftaar, T.M. (2018, January 6\u20139). The Effects of Random Undersampling with Simulated Class Imbalance for Big Data. Proceedings of the IEEE International Conference on Information Reuse and Integration (IRI), Salt Lake City, UT, USA.","DOI":"10.1109\/IRI.2018.00018"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Huang, H., Xu, C., Yoo, S., Yan, W., Wang, T., and Xue, F. (2020, January 19\u201323). Imbalanced Time Series Classification for Flight Data Analyzing with Nonlinear Granger Causality Learning. Proceedings of the 29th ACM International Conference on Information & Knowledge Management, New York, NY, USA.","DOI":"10.1145\/3340531.3412710"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1016\/j.artmed.2005.03.002","article-title":"Learning from Imbalanced Data in Surveillance of Nosocomial Infection","volume":"37","author":"Cohen","year":"2006","journal-title":"Artif. Intell. Med."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Tsinganos, P., Cornelis, B., Cornelis, J., Jansen, B., and Skodras, A. (2020). Data Augmentation of Surface Electromyography for Hand Gesture Recognition. Sensors, 20.","DOI":"10.3390\/s20174892"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Lopez-Nava, I.H., Valent\u00edn-Coronado, L.M., Garcia-Constantino, M., and Favela, J. (2020). Gait Activity Classification on Unbalanced Data from Inertial Sensors Using Shallow and Deep Learning. Sensors, 20.","DOI":"10.3390\/s20174756"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1109\/LSP.2017.2657381","article-title":"Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification","volume":"24","author":"Salamon","year":"2017","journal-title":"IEEE Signal. Process. Lett."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Wang, F., Zhong, S.-H., Peng, J., Jiang, J., and Liu, Y. (2018, January 5\u20137). Data Augmentation for EEG-Based Emotion Recognition with Deep Convolutional Neural Networks. Proceedings of the 24th International Conference on Multimedia Modeling, Bangkok, Thailand.","DOI":"10.1007\/978-3-319-73600-6_8"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Krell, M.M., and Kim, S.K. (2017, January 11\u201315). Rotational Data Augmentation for Electroencephalographic Data. Proceedings of the 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Jeju Island, Korea.","DOI":"10.1109\/EMBC.2017.8036864"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Iwana, B.K., and Uchida, S. (2021). An Empirical Survey of Data Augmentation for Time Series Classification with Neural Networks. PLoS ONE, 16.","DOI":"10.1371\/journal.pone.0254841"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1109\/TCDS.2020.2965166","article-title":"Improving the Generalization Ability of Deep Neural Networks for Cross-Domain Visual Recognition","volume":"13","author":"Zheng","year":"2021","journal-title":"IEEE Trans. Cogn. Dev. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ifacol.2019.12.406","article-title":"Generative Adversarial Network Based Image Augmentation for Insect Pest Classification Enhancement","volume":"Volume 52","author":"Lu","year":"2019","journal-title":"Proceedings of the IFAC PapersOnLine: 6th IFAC Conference on Sensing, Control and Automation Technologies for Agriculture"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"212","DOI":"10.1016\/j.neucom.2019.06.043","article-title":"MFC-GAN: Class-Imbalanced Dataset Classification Using Multiple Fake Class Generative Adversarial Network","volume":"361","author":"Elyan","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_29","unstructured":"Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014, January 8\u201313). Generative Adversarial Nets. Proceedings of the 27th International Conference on Neural Information Processing Systems (NIPS\u201914)- Volume 2, Montreal, QC, Canada."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Thanh-Tung, H., and Tran, T. (2020, January 19\u201324). Catastrophic Forgetting and Mode Collapse in GANs. Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, UK.","DOI":"10.1109\/IJCNN48605.2020.9207181"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Adiga, S., Attia, M.A., Chang, W.T., and Tandon, R. (2018, January 26\u201328). On the Tradeoff between Mode Collapse and Sample Quality in Generative Adversarial Networks. Proceedings of the Global Conference on Signal and Information Processing (GlobalSIP 2018), Anaheim, CA, USA.","DOI":"10.1109\/GlobalSIP.2018.8646478"},{"key":"ref_32","unstructured":"Lala, S., Shady, M., Belyaeva, A., and Liu, M. (2018, January 25\u201327). Evaluation of Mode Collapse in Generative Adversarial Networks. Proceedings of the High Performance Extreme Computing Conference (HPEC \u201918), Waltham, MA, USA."},{"key":"ref_33","unstructured":"Metz, L., Poole, B., Pfau, D., and Sohl-Dickstein, J. (2017, January 24\u201326). Unrolled Generative Adversarial Networks. Proceedings of the 5th International Conference on Learning Representations, Toulon, France."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"464","DOI":"10.1016\/j.eswa.2017.09.030","article-title":"Effective Data Generation for Imbalanced Learning Using Conditional Generative Adversarial Networks","volume":"91","author":"Douzas","year":"2018","journal-title":"Expert Syst. Appl."},{"key":"ref_35","unstructured":"Donahue, C., McAuley, J., and Puckette, M. (May, January 30). Synthesizing Audio with GANs. Proceedings of the 6th International Conference on Learning Representations (Workshop), Vancouver, BC, Canada."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Madhu, A., and Kumaraswamy, S. (2019, January 2\u20136). Data Augmentation Using Generative Adversarial Network for Environmental Sound Classification. Proceedings of the 27th European Signal Processing Conference (EUSIPCO), La Coru\u00f1a, Spain.","DOI":"10.23919\/EUSIPCO.2019.8902819"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Setiawan, B.D., Kryssanov, V., and Serd\u00fclt, U.I. (2021, January 17\u201320). Smartphone Sensor Data Augmentation for Automatic Road Surface Assessment Using a Small Training Dataset. Proceedings of the 2021 IEEE International Conference on Big Data and Smart Computing (BigComp), Jeju, South Korea.","DOI":"10.1109\/BigComp51126.2021.00052"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Jenni, S., and Favaro, P. (2019, January 15\u201320). On Stabilizing Generative Adversarial Training with Noise. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01242"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"519","DOI":"10.1007\/s00779-019-01234-z","article-title":"A Deep Learning Approach to Automatic Road Surface Monitoring and Pothole Detection","volume":"24","author":"Varona","year":"2020","journal-title":"Pers. Ubiquitous Comput."},{"key":"ref_40","unstructured":"Kingma, D.P., and Ba, J.L. (2015, January 7\u20139). Adam: A Method for Stochastic Optimization. Proceedings of the 3rd International Conference on Learning Representations, San Diego, CA, USA."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1007\/s11045-020-00731-2","article-title":"Label-Preserving Data Augmentation for Mobile Sensor Data","volume":"32","author":"Kim","year":"2021","journal-title":"Multidimens. Syst. Signal. Process."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Zantalis, F., Koulouras, G., Karabetsos, S., and Kandris, D. (2019). A Review of Machine Learning and IoT in Smart Transportation. Futur. Internet, 11.","DOI":"10.3390\/fi11040094"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1109\/MITS.2014.2328673","article-title":"Driver Behavior Profiling Using Smartphones: A Low-Cost Platform for Driver Monitoring","volume":"7","author":"Castignani","year":"2015","journal-title":"IEEE Intell. Transp. Syst. Mag."},{"key":"ref_44","unstructured":"Li, F., Zhang, H., Che, H., and Qiu, X. (2016, January 1\u20134). Dangerous Driving Behavior Detection Using Smartphone Sensors. Proceedings of the Conference on Intelligent Transportation Systems, Proceedings (ITSC), Rio de Janeiro, Brazil."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1007\/s11036-011-0304-8","article-title":"WreckWatch: Automatic Traffic Accident Detection and Notification with Smartphones","volume":"16","author":"White","year":"2011","journal-title":"Mob. Networks Appl."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"4231","DOI":"10.1109\/JSEN.2017.2702739","article-title":"RoadSense: Smartphone Application to Estimate Road Conditions Using Accelerometer and Gyroscope","volume":"17","author":"Allouch","year":"2017","journal-title":"IEEE Sens. J."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"El-Kady, A., Emara, K., Eleliemy, M.H., and Shaaban, E. (2019, January 8\u201310). Road Surface Quality Detection Using Smartphone Sensors: Egyptian Roads Case Study. Proceedings of the 2019 9th International Conference on Intelligent Computing and Information Systems (ICICIS), Cairo, Egypt.","DOI":"10.1109\/ICICIS46948.2019.9014721"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Eriksson, J., Girod, L., Hull, B., Newton, R., Madden, S., and Balakrishnan, H. (2008, January 17\u201320). The Pothole Patrol: Using a Mobile Sensor Network for Road Surface Monitoring. Proceedings of the 6th International Conference on Mobile Systems, Applications, and Services, Breckenridge, CO, USA.","DOI":"10.1145\/1378600.1378605"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Park, J., Min, K., Kim, H., Lee, W., Cho, G., and Huh, K. (2018). Road Surface Classification Using a Deep Ensemble Network with Sensor Feature Selection. Sensors, 18.","DOI":"10.3390\/s18124342"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Lee, T., Chun, C., and Ryu, S.-K. (2021). Detection of Road-Surface Anomalies Using a Smartphone Camera and Accelerometer. Sensors, 21.","DOI":"10.3390\/s21020561"},{"key":"ref_51","unstructured":"Maas, A.L., Hannun, A.Y., and Ng, A.Y. (2013, January 16\u201321). Rectifier Nonlinearities Improve Neural Network Acoustic Models. Proceedings of the 30th International Conference on Machine Learning, Atlanta, GA, USA."},{"key":"ref_52","unstructured":"Glorot, X., Bordes, A., and Bengio, Y. (2011, January 11\u201313). Deep Sparse Rectifier Neural Networks. Proceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS), Fort Lauderdale, FL, USA."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"525","DOI":"10.1007\/s11831-019-09388-y","article-title":"Applications of Generative Adversarial Networks (GANs): An Updated Review","volume":"28","author":"Alqahtani","year":"2021","journal-title":"Arch. Comput. Methods Eng."},{"key":"ref_54","unstructured":"Arora, S., Ge, R., Liang, Y., Ma, T., and Zhang, Y. (2017, January 6\u201311). Generalization and Equilibrium in Generative Adversarial Nets (GANs). Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_55","unstructured":"Dau, H.A., Eamonn, K., Kamgar, K., Yeh, C.-C.M., Zhu, Y., Gharghabi, S., Ratanamahatana, C.A., Chen, Y., Hu, B., and Begum, N. (2021, May 31). The UCR Time Series Classification Archive. Available online: https:\/\/www.cs.ucr.edu\/~eamonn\/time_series_data_2018\/."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1109\/MSP.2017.2765202","article-title":"Generative Adversarial Networks: An Overview","volume":"35","author":"Creswell","year":"2018","journal-title":"IEEE Signal. Process. Mag."},{"key":"ref_57","unstructured":"Lee, J.D., Simchowitz, M., Jordan, M.I., and Recht, B. (2016, January 23\u201326). Gradient Descent Only Converges to Minimizers. Proceedings of the 29th Annual Conference on Learning Theory, New York, NY, USA."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/20\/6892\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:16:54Z","timestamp":1760167014000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/20\/6892"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,18]]},"references-count":57,"journal-issue":{"issue":"20","published-online":{"date-parts":[[2021,10]]}},"alternative-id":["s21206892"],"URL":"https:\/\/doi.org\/10.3390\/s21206892","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,18]]}}}