{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,11]],"date-time":"2026-04-11T21:26:46Z","timestamp":1775942806090,"version":"3.50.1"},"reference-count":46,"publisher":"MDPI AG","issue":"14","license":[{"start":{"date-parts":[[2023,7,18]],"date-time":"2023-07-18T00:00:00Z","timestamp":1689638400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Institute for Information and Communications Technology Promotion (IITP)","award":["2017-0-00655"],"award-info":[{"award-number":["2017-0-00655"]}]},{"name":"Institute for Information and Communications Technology Promotion (IITP)","award":["NRF-2023R1A2C100358511"],"award-info":[{"award-number":["NRF-2023R1A2C100358511"]}]},{"name":"National Research Foundation of Korea (NRF)","award":["2017-0-00655"],"award-info":[{"award-number":["2017-0-00655"]}]},{"name":"National Research Foundation of Korea (NRF)","award":["NRF-2023R1A2C100358511"],"award-info":[{"award-number":["NRF-2023R1A2C100358511"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Human Activity Recognition (HAR) has gained significant attention due to its broad range of applications, such as healthcare, industrial work safety, activity assistance, and driver monitoring. Most prior HAR systems are based on recorded sensor data (i.e., past information) recognizing human activities. In fact, HAR works based on future sensor data to predict human activities are rare. Human Activity Prediction (HAP) can benefit in multiple applications, such as fall detection or exercise routines, to prevent injuries. This work presents a novel HAP system based on forecasted activity data of Inertial Measurement Units (IMU). Our HAP system consists of a deep learning forecaster of IMU activity signals and a deep learning classifier to recognize future activities. Our deep learning forecaster model is based on a Sequence-to-Sequence structure with attention and positional encoding layers. Then, a pre-trained deep learning Bi-LSTM classifier is used to classify future activities based on the forecasted IMU data. We have tested our HAP system for five daily activities with two tri-axial IMU sensors. The forecasted signals show an average correlation of 91.6% to the actual measured signals of the five activities. The proposed HAP system achieves an average accuracy of 97.96% in predicting future activities.<\/jats:p>","DOI":"10.3390\/s23146491","type":"journal-article","created":{"date-parts":[[2023,7,19]],"date-time":"2023-07-19T01:02:23Z","timestamp":1689728543000},"page":"6491","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["Human Activity Prediction Based on Forecasted IMU Activity Signals by Sequence-to-Sequence Deep Neural Networks"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7583-1146","authenticated-orcid":false,"given":"Ismael Espinoza","family":"Jaramillo","sequence":"first","affiliation":[{"name":"Department of Electronics and Information Convergence Engineering, Kyung Hee University, Yongin 17104, Republic of Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7509-9354","authenticated-orcid":false,"given":"Channabasava","family":"Chola","sequence":"additional","affiliation":[{"name":"Department of Electronics and Information Convergence Engineering, Kyung Hee University, Yongin 17104, Republic of Korea"}]},{"given":"Jin-Gyun","family":"Jeong","sequence":"additional","affiliation":[{"name":"Department of Electronics and Information Convergence Engineering, Kyung Hee University, Yongin 17104, Republic of Korea"}]},{"given":"Ji-Heon","family":"Oh","sequence":"additional","affiliation":[{"name":"Department of Electronics and Information Convergence Engineering, Kyung Hee University, Yongin 17104, Republic of Korea"}]},{"given":"Hwanseok","family":"Jung","sequence":"additional","affiliation":[{"name":"Department of Electronics and Information Convergence Engineering, Kyung Hee University, Yongin 17104, Republic of Korea"}]},{"given":"Jin-Hyuk","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Electronics and Information Convergence Engineering, Kyung Hee University, Yongin 17104, Republic of Korea"}]},{"given":"Won Hee","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Software Convergence, Kyung Hee University, Yongin 17104, Republic of Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7118-1708","authenticated-orcid":false,"given":"Tae-Seong","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of Electronics and Information Convergence Engineering, Kyung Hee University, Yongin 17104, Republic of Korea"}]}],"member":"1968","published-online":{"date-parts":[[2023,7,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"5223","DOI":"10.1109\/JBHI.2022.3193148","article-title":"Dual-Branch Interactive Networks on Multichannel Time Series for Human Activity Recognition","volume":"26","author":"Tang","year":"2022","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_2","unstructured":"Yeh, Y.H., Wong, D.P.Y., Lee, C.T., and Chou, P.H. (2022). IVSP\u201922: Proceedings of the 2022 4th International Conference on Image, Video and Signal Processing, Singapore, 18\u201320 March 2022, Association for Computing Machinery."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"5241","DOI":"10.1007\/s11227-021-04065-z","article-title":"Performance-Enhanced Real-Time Lifestyle Tracking Model Based on Human Activity Recognition (PERT-HAR) Model through Smartphones","volume":"78","author":"Ishwarya","year":"2022","journal-title":"J. Supercomput."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1068","DOI":"10.1109\/JSYST.2014.2351491","article-title":"Lower Limb Wearable Robots for Assistance and Rehabilitation: A State of the Art","volume":"10","author":"Huo","year":"2016","journal-title":"IEEE Syst. J."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Fernandes, C., Matos, L.M., Folgado, D., Nunes, M.L., Pereira, J.R., Pilastri, A., and Cortez, P. (2022, January 18\u201323). A Deep Learning Approach to Prevent Problematic Movements of Industrial Workers Based on Inertial Sensors. Proceedings of the International Joint Conference on Neural Networks, Padua, Italy.","DOI":"10.1109\/IJCNN55064.2022.9892409"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"85154","DOI":"10.1109\/ACCESS.2022.3197618","article-title":"Future Activities Prediction Framework in Smart Homes Environment","volume":"10","author":"Mohamed","year":"2022","journal-title":"IEEE Access"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"102738","DOI":"10.1016\/j.jnca.2020.102738","article-title":"A Review and Categorization of Techniques on Device-Free Human Activity Recognition","volume":"167","author":"Hussain","year":"2020","journal-title":"J. Netw. Comput. Appl."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"12793","DOI":"10.1007\/s00521-023-08374-7","article-title":"Comprehensive Machine and Deep Learning Analysis of Sensor-Based Human Activity Recognition","volume":"35","author":"Balaha","year":"2023","journal-title":"Neural Comput. Appl."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1016\/j.future.2017.11.029","article-title":"A Robust Human Activity Recognition System Using Smartphone Sensors and Deep Learning","volume":"81","author":"Hassan","year":"2018","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"106970","DOI":"10.1016\/j.knosys.2021.106970","article-title":"A Review of Multimodal Human Activity Recognition with Special Emphasis on Classification, Applications, Challenges and Future Directions","volume":"223","author":"Yadav","year":"2021","journal-title":"Knowl. Based Syst."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Kim, Y.W., Joa, K.L., Jeong, H.Y., and Lee, S. (2021). Wearable Imu-Based Human Activity Recognition Algorithm for Clinical Balance Assessment Using 1d-Cnn and Gru Ensemble Model. Sensors, 21.","DOI":"10.3390\/s21227628"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Anagnostis, A., Benos, L., Tsaopoulos, D., Tagarakis, A., Tsolakis, N., and Bochtis, D. (2021). Human Activity Recognition through Recurrent Neural Networks for Human\u2013Robot Interaction in Agriculture. Appl. Sci., 11.","DOI":"10.3390\/app11052188"},{"key":"ref_13","first-page":"1","article-title":"Deep Learning for Sensor-Based Human Activity Recognition: Overview, Challenges, and Opportunities","volume":"54","author":"Chen","year":"2021","journal-title":"ACM Comput. Surv."},{"key":"ref_14","first-page":"420","article-title":"HARD-Net: Hardness-AwaRe Discrimination Network for 3D Early Activity Prediction","volume":"Volume 12356","author":"Li","year":"2020","journal-title":"Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Xu, W., Yu, J., Miao, Z., Wan, L., and Ji, Q. (2019, January 21\u201325). Prediction-CGAN: Human Action Prediction with Conditional Generative Adversarial Networks. Proceedings of the MM 2019\u2014Proceedings of the 27th ACM International Conference on Multimedia, Nice, France.","DOI":"10.1145\/3343031.3351073"},{"key":"ref_16","unstructured":"Furnari, A., and Farinella, G.M. (November, January 27). What Would You Expect? Anticipating Egocentric Actions with Rolling-Unrolling LSTMs and Modality Attention. Proceedings of the ICCV Open Access by Computer Vision Foundation, Seoul, Republic of Korea."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Farha, Y.A., Richard, A., and Gall, J. (2018, January 18\u201322). When Will You Do What?\u2014Anticipating Temporal Occurrences of Activities. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00560"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"133190","DOI":"10.1109\/ACCESS.2019.2940729","article-title":"Smartphone, and Smartwatch-Based Biometrics Using Activities of Daily Living","volume":"7","author":"Weiss","year":"2019","journal-title":"IEEE Access"},{"key":"ref_19","first-page":"91","article-title":"MHealthDroid: A Novel Framework for Agile Development of Mobile Health Applications","volume":"Volume 8868","author":"Banos","year":"2014","journal-title":"Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1109\/TII.2022.3165875","article-title":"Multi-ResAtt: Multilevel Residual Network with Attention for Human Activity Recognition Using Wearable Sensors","volume":"19","author":"Dahou","year":"2023","journal-title":"IEEE Trans Ind. Inf."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Kim, T., Park, J., Lee, J., and Park, J. (2021). Predicting Human Motion Signals Using Modern Deep Learning Techniques and Smartphone Sensors. Sensors, 21.","DOI":"10.3390\/s21248270"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"105905","DOI":"10.1016\/j.compbiomed.2022.105905","article-title":"Real-Time Forecasting of Exercise-Induced Fatigue from Wearable Sensors","volume":"148","author":"Jiang","year":"2022","journal-title":"Comput. Biol. Med."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Soleimani, R., and Lobaton, E. (2022). Enhancing Inference on Physiological and Kinematic Periodic Signals via Phase-Based Interpretability and Multi-Task Learning. Information, 13.","DOI":"10.3390\/info13070326"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1421","DOI":"10.1007\/s10994-019-05815-0","article-title":"Temporal Pattern Attention for Multivariate Time Series Forecasting","volume":"108","author":"Shih","year":"2019","journal-title":"Mach. Learn."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1145\/3559540","article-title":"Generative Adversarial Networks in Time Series: A Systematic Literature Review","volume":"55","author":"Brophy","year":"2023","journal-title":"ACM Comput. Surv."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Hazra, D., and Byun, Y.C. (2020). SynSigGAN: Generative Adversarial Networks for Synthetic Biomedical Signal Generation. Biology, 9.","DOI":"10.3390\/biology9120441"},{"key":"ref_27","first-page":"1","article-title":"Time-Series Generative Adversarial Networks","volume":"32","author":"Yoon","year":"2019","journal-title":"Adv. Neural. Inf. Process. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Sumiya, Y., Horie, K., Shiokawa, H., and Kitagawa, H. (2019, January 17\u201319). Nr-GAN: Noise Reduction GaN for Mice Electroencephalogram Signals. Proceedings of the ICBSP\u201919: Proceedings of the 2019 4th International Conference on Biomedical Imaging, Signal Processing, Nagoya, Japan.","DOI":"10.1145\/3366174.3366186"},{"key":"ref_29","unstructured":"Hyland, S.L., Zurich, E., Esteban, C., and R\u00e4tsch ETH Zurich, G. (2017). Real-Valued (Medical) Time Series Generation with Recurrent Conditional GANs. arXiv."},{"key":"ref_30","first-page":"133","article-title":"TTS-GAN: A Transformer-Based Time-Series Generative Adversarial Network","volume":"Volume 13263","author":"Li","year":"2022","journal-title":"Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Jaramillo, I.E., Jeong, J.G., Lopez, P.R., Lee, C.H., Kang, D.Y., Ha, T.J., Oh, J.H., Jung, H., Lee, J.H., and Lee, W.H. (2022). Real-Time Human Activity Recognition with IMU and Encoder Sensors in Wearable Exoskeleton Robot via Deep Learning Networks. Sensors, 22.","DOI":"10.3390\/s22249690"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Reiss, A., and Stricker, D. (2012, January 18\u201322). Introducing a New Benchmarked Dataset for Activity Monitoring. Proceedings of the International Symposium on Wearable Computers, ISWC, Newcastle, UK.","DOI":"10.1109\/ISWC.2012.13"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Luwe, Y.J., Lee, C.P., and Lim, K.M. (2022). Wearable Sensor-Based Human Activity Recognition with Hybrid Deep Learning Model. Informatics, 9.","DOI":"10.3390\/informatics9030056"},{"key":"ref_34","unstructured":"Guennec, A.L., Malinowski, S., and Tavenard, R. (2016, January 19\u201323). Data Augmentation for Time Series Classification Using Convolutional Neural Networks. Proceedings of the ECML\/PKDD Workshop on Advanced Analytics and Learning on Temporal Data, Riva Del Garda, Italy."},{"key":"ref_35","first-page":"5999","article-title":"Attention Is All You Need","volume":"30","author":"Vaswani","year":"2017","journal-title":"Adv. Neural. Inf. Process. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"246","DOI":"10.1016\/j.neucom.2019.07.034","article-title":"Multi-Head CNN\u2013RNN for Multi-Time Series Anomaly Detection: An Industrial Case Study","volume":"363","author":"Canizo","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_37","first-page":"3104","article-title":"Sequence to Sequence Learning with Neural Networks","volume":"4","author":"Sutskever","year":"2014","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"3443","DOI":"10.1007\/s11063-022-10774-0","article-title":"Long Time Series Deep Forecasting with Multiscale Feature Extraction and Seq2seq Attention Mechanism","volume":"54","author":"Wang","year":"2022","journal-title":"Neural Process. Lett."},{"key":"ref_39","unstructured":"Wen, Q., Zhou, T., Zhang, C., Chen, W., Ma, Z., Yan, J., and Sun, L. (2022). Transformers in Time Series: A Survey. arXiv."},{"key":"ref_40","unstructured":"Gehring, J., Auli, M., Grangier, D., Yarats, D., and Dauphin, Y.N. (2017, January 6\u201311). Convolutional Sequence to Sequence Learning. Proceedings of the International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1366","DOI":"10.1007\/s11263-022-01594-9","article-title":"Human Action Recognition and Prediction: A Survey","volume":"130","author":"Kong","year":"2022","journal-title":"Int. J. Comput. Vis."},{"key":"ref_42","unstructured":"Bahdanau, D., Cho, K.H., and Bengio, Y. (2015, January 7\u20139). Neural Machine Translation by Jointly Learning to Align and Translate. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015\u2014Conference Track Proceedings, San Diego, CA, USA."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Lee, S.H., Lee, D.W., and Kim, M.S. (2023). A Deep Learning-Based Semantic Segmentation Model Using MCNN and Attention Layer for Human Activity Recognition. Sensors, 23.","DOI":"10.3390\/s23042278"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"2106","DOI":"10.1109\/TIE.2022.3161812","article-title":"Multiscale Deep Feature Learning for Human Activity Recognition Using Wearable Sensors","volume":"70","author":"Tang","year":"2023","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"4095","DOI":"10.1007\/s00371-021-02283-3","article-title":"A Multibranch CNN-BiLSTM Model for Human Activity Recognition Using Wearable Sensor Data","volume":"38","author":"Challa","year":"2022","journal-title":"Vis. Comput."},{"key":"ref_46","first-page":"1","article-title":"Deep-HAR: An Ensemble Deep Learning Model for Recognizing the Simple, Complex, and Heterogeneous Human Activities","volume":"2023","author":"Kumar","year":"2023","journal-title":"Multimed. Tools Appl."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/14\/6491\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:14:13Z","timestamp":1760127253000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/14\/6491"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,18]]},"references-count":46,"journal-issue":{"issue":"14","published-online":{"date-parts":[[2023,7]]}},"alternative-id":["s23146491"],"URL":"https:\/\/doi.org\/10.3390\/s23146491","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,18]]}}}