{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,24]],"date-time":"2026-04-24T13:30:31Z","timestamp":1777037431472,"version":"3.51.4"},"reference-count":29,"publisher":"MDPI AG","issue":"24","license":[{"start":{"date-parts":[[2022,12,10]],"date-time":"2022-12-10T00:00:00Z","timestamp":1670630400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Development of AI-based Embedded Motion Recognizer","award":["20211680"],"award-info":[{"award-number":["20211680"]}]},{"name":"Development of AI-based Embedded Motion Recognizer","award":["20201806"],"award-info":[{"award-number":["20201806"]}]},{"name":"Development of AI-based Motion Recognition Technology","award":["20211680"],"award-info":[{"award-number":["20211680"]}]},{"name":"Development of AI-based Motion Recognition Technology","award":["20201806"],"award-info":[{"award-number":["20201806"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Wearable exoskeleton robots have become a promising technology for supporting human motions in multiple tasks. Activity recognition in real-time provides useful information to enhance the robot\u2019s control assistance for daily tasks. This work implements a real-time activity recognition system based on the activity signals of an inertial measurement unit (IMU) and a pair of rotary encoders integrated into the exoskeleton robot. Five deep learning models have been trained and evaluated for activity recognition. As a result, a subset of optimized deep learning models was transferred to an edge device for real-time evaluation in a continuous action environment using eight common human tasks: stand, bend, crouch, walk, sit-down, sit-up, and ascend and descend stairs. These eight robot wearer\u2019s activities are recognized with an average accuracy of 97.35% in real-time tests, with an inference time under 10 ms and an overall latency of 0.506 s per recognition using the selected edge device.<\/jats:p>","DOI":"10.3390\/s22249690","type":"journal-article","created":{"date-parts":[[2022,12,12]],"date-time":"2022-12-12T05:10:19Z","timestamp":1670821819000},"page":"9690","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":51,"title":["Real-Time Human Activity Recognition with IMU and Encoder Sensors in Wearable Exoskeleton Robot via Deep Learning Networks"],"prefix":"10.3390","volume":"22","author":[{"given":"Ismael Espinoza","family":"Jaramillo","sequence":"first","affiliation":[{"name":"Department of Electronics and Information Convergence Engineering, Kyung Hee University, Yongin 17104, Republic of Korea"}]},{"given":"Jin Gyun","family":"Jeong","sequence":"additional","affiliation":[{"name":"Department of Electronics and Information Convergence Engineering, Kyung Hee University, Yongin 17104, Republic of Korea"}]},{"given":"Patricio Rivera","family":"Lopez","sequence":"additional","affiliation":[{"name":"AI Laboratory, DeltaX, Seoul 04522, Republic of Korea"}]},{"given":"Choong-Ho","family":"Lee","sequence":"additional","affiliation":[{"name":"Hyundai Rotem, Uiwang-si 16082, Republic of Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5074-3716","authenticated-orcid":false,"given":"Do-Yeon","family":"Kang","sequence":"additional","affiliation":[{"name":"Hyundai Rotem, Uiwang-si 16082, Republic of Korea"}]},{"given":"Tae-Jun","family":"Ha","sequence":"additional","affiliation":[{"name":"Hyundai Rotem, Uiwang-si 16082, Republic of Korea"}]},{"given":"Ji-Heon","family":"Oh","sequence":"additional","affiliation":[{"name":"Department of Electronics and Information Convergence Engineering, Kyung Hee University, Yongin 17104, Republic of Korea"}]},{"given":"Hwanseok","family":"Jung","sequence":"additional","affiliation":[{"name":"Department of Electronics and Information Convergence Engineering, Kyung Hee University, Yongin 17104, Republic of Korea"}]},{"given":"Jin Hyuk","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Electronics and Information Convergence Engineering, Kyung Hee University, Yongin 17104, Republic of Korea"}]},{"given":"Won Hee","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Software Convergence, Kyung Hee University, Yongin 17104, Republic of Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7118-1708","authenticated-orcid":false,"given":"Tae-Seong","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of Electronics and Information Convergence Engineering, Kyung Hee University, Yongin 17104, Republic of Korea"}]}],"member":"1968","published-online":{"date-parts":[[2022,12,10]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1068","DOI":"10.1109\/JSYST.2014.2351491","article-title":"Lower Limb Wearable Robots for Assistance and Rehabilitation: A State of the Art","volume":"10","author":"Huo","year":"2016","journal-title":"IEEE Syst. J."},{"key":"ref_2","first-page":"77","article-title":"Deep Learning for Sensor-Based Human Activity Recognition: Overview, Challenges, and Opportunities","volume":"54","author":"Chen","year":"2021","journal-title":"ACM Comput. Surv."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"671","DOI":"10.1080\/00140139.2015.1081988","article-title":"Exoskeletons for Industrial Application and Their Potential Effects on Physical Work Load","volume":"59","author":"Bosch","year":"2016","journal-title":"Ergonomics"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"012020","DOI":"10.1088\/1757-899X\/1140\/1\/012020","article-title":"Inertial Measurement Unit Based Human Action Recognition for Soft-Robotic Exoskeleton","volume":"Volume 1140","author":"Kuschan","year":"2021","journal-title":"IOP Conference Series: Materials Science and Engineering"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1743-0003-12-1","article-title":"Control Strategies for Active Lower Extremity Prosthetics and Orthotics: A Review","volume":"12","author":"Tucker","year":"2015","journal-title":"J. Neuroeng. Rehabil."},{"key":"ref_6","unstructured":"Schmidt, P., Reiss, A., Hanselmann, M., and Stiefelhagen, R. (2017, January 11\u201315). CNN-Based Sensor Fusion Techniques for Multimodal Human Activity Recognition. Proceedings of the 2017 ACM International Symposium on Wearable Computers, Maui, HI, USA."},{"key":"ref_7","first-page":"800","article-title":"2D Human Pose Estimation Based on Object Detection Using RGB-D Information","volume":"12","author":"Park","year":"2018","journal-title":"KSII Trans. Internet Inf. Syst."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"3047","DOI":"10.1109\/TNNLS.2019.2935173","article-title":"Graph Edge Convolutional Neural Networks for Skeleton-Based Action Recognition","volume":"31","author":"Zhang","year":"2020","journal-title":"IEEE Trans. Neural. Netw. Learn. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"107561","DOI":"10.1016\/j.patcog.2020.107561","article-title":"Sensor-Based and Vision-Based Human Activity Recognition: A Comprehensive Survey","volume":"108","author":"Min","year":"2020","journal-title":"Pattern Recognit."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.patrec.2018.02.010","article-title":"Deep Learning for Sensor-Based Activity Recognition: A Survey","volume":"119","author":"Wang","year":"2019","journal-title":"Pattern Recognit. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2152006","DOI":"10.1142\/S0218001421520066","article-title":"An Integrated ARMA-Based Deep Autoencoder and GRU Classifier System for Enhanced Recognition of Daily Hand Activities","volume":"35","author":"Rivera","year":"2021","journal-title":"Int. J. Pattern Recognit. Artif. Intell."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"887","DOI":"10.1007\/s00779-011-0403-3","article-title":"A Single Tri-Axial Accelerometer-Based Real-Time Personal Life Log System Capable of Human Activity Recognition and Exercise Information Generation","volume":"15","author":"Lee","year":"2011","journal-title":"Pers. Ubiquitous Comput."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1271","DOI":"10.1007\/s11517-010-0701-3","article-title":"Accelerometer\u2019s Position Independent Physical Activity Recognition System for Long-Term Activity Monitoring in the Elderly","volume":"48","author":"Khan","year":"2010","journal-title":"Med. Biol. Eng. Comput."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1166","DOI":"10.1109\/TITB.2010.2051955","article-title":"A Triaxial Accelerometer-Based Physical-Activity Recognition via Augmented-Signal Features and a Hierarchical Recognizer","volume":"14","author":"Khan","year":"2010","journal-title":"IEEE Trans. Inf. Technol. Biomed."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1309","DOI":"10.1109\/JSEN.2021.3069927","article-title":"Human Activity Recognition with Smartphone and Wearable Sensors Using Deep Learning Techniques: A Review","volume":"21","author":"Ramanujam","year":"2021","journal-title":"IEEE Sens. J."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Kim, Y.W., Joa, K.L., Jeong, H.Y., and Lee, S. (2021). Wearable Imu-Based Human Activity Recognition Algorithm for Clinical Balance Assessment Using 1d-Cnn and Gru Ensemble Model. Sensors, 21.","DOI":"10.3390\/s21227628"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Jantawong, P., Hnoohom, N., Jitpattanakul, A., and Mekruksavanich, S. (2021, January 18\u201320). A Lightweight Deep Learning Network for Sensor-Based Human Activity Recognition Using IMU Sensors of a Low-Power Wearable Device. Proceedings of the ICSEC 2021\u201425th International Computer Science and Engineering Conference, Chiang Rai, Thailand.","DOI":"10.1109\/ICSEC53205.2021.9684631"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Sherratt, F., Plummer, A., and Iravani, P. (2021). Understanding Lstm Network Behaviour of Imu-Based Locomotion Mode Recognition for Applications in Prostheses and Wearables. Sensors, 21.","DOI":"10.3390\/s21041264"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Fu, Z., He, X., Wang, E., Huo, J., Huang, J., and Wu, D. (2021). Personalized Human Activity Recognition Based on Integrated Wearable Sensor and Transfer Learning. Sensors, 21.","DOI":"10.3390\/s21030885"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Mascret, Q., Bielmann, M., Fall, C.-L., Bouyer, L.J., and Gosselin, B. (2018, January 18\u201321). Real-Time Human Physical Activity Recognition with Low Latency Prediction Feedback Using Raw IMU Data. Proceedings of the 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Honolulu, HI, USA.","DOI":"10.1109\/EMBC.2018.8512252"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"5223","DOI":"10.1109\/JBHI.2022.3193148","article-title":"Dual-Branch Interactive Networks on Multichannel Time Series for Human Activity Recognition","volume":"26","author":"Tang","year":"2022","journal-title":"IEEE J. Biomed. Health Inf."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Yeh, Y.H., Wong, D.P.Y., Lee, C.T., and Chou, P.H. (2022, January 18\u201320). Deep Learning-Based Real-Time Activity Recognition with Multiple Inertial Sensors. Proceedings of the ACM International Conference Proceeding Series, Singapore.","DOI":"10.1145\/3531232.3531245"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TIM.2022.3216413","article-title":"Deep Convolutional Networks with Tunable Speed-Accuracy Tradeoff for Human Activity Recognition Using Wearables","volume":"71","author":"Wang","year":"2022","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1827","DOI":"10.1109\/TNSRE.2021.3107780","article-title":"Real-Time Activity Recognition with Instantaneous Characteristic Features of Thigh Kinematics","volume":"29","author":"Cheng","year":"2021","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_25","unstructured":"le Guennec, A., Malinowski, S., and Tavenard, R. (2016, January 1). Data Augmentation for Time Series Classification Using Convolutional Neural Networks. Proceedings of the ECML\/PKDD Workshop on Advanced Analytics and Learning on Temporal Data, Riva Del Garda, Italy."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Banjarey, K., Prakash Sahu, S., and Kumar Dewangan, D. (2021, January 8\u201310). A Survey on Human Activity Recognition Using Sensors and Deep Learning Methods. Proceedings of the 2021 5th International Conference on Computing Methodologies and Communication (ICCMC), Erode, India.","DOI":"10.1109\/ICCMC51019.2021.9418255"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1113","DOI":"10.1111\/coin.12318","article-title":"A Study of Deep Neural Networks for Human Activity Recognition","volume":"36","author":"Sansano","year":"2020","journal-title":"Comput. Intell."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Zhang, S., Li, Y., Zhang, S., Shahabi, F., Xia, S., Deng, Y., and Alshurafa, N. (2022). Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on Advances. Sensors, 22.","DOI":"10.3390\/s22041476"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1507","DOI":"10.1007\/s13369-021-06008-5","article-title":"A Comparative Study on Classifying Human Activities Using Classical Machine and Deep Learning Methods","volume":"47","author":"Bozkurt","year":"2022","journal-title":"Arab. J. Sci. Eng."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/24\/9690\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:37:56Z","timestamp":1760146676000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/24\/9690"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,10]]},"references-count":29,"journal-issue":{"issue":"24","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["s22249690"],"URL":"https:\/\/doi.org\/10.3390\/s22249690","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,10]]}}}