{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T16:24:07Z","timestamp":1778603047355,"version":"3.51.4"},"reference-count":44,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2023,5,12]],"date-time":"2023-05-12T00:00:00Z","timestamp":1683849600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Deanship of Scientific Research at Najran University","award":["NU\/RG\/SERC\/12\/40"],"award-info":[{"award-number":["NU\/RG\/SERC\/12\/40"]}]},{"name":"Deanship of Scientific Research at Najran University","award":["PNURSP2023R440"],"award-info":[{"award-number":["PNURSP2023R440"]}]},{"name":"Deanship of Scientific Research at Najran University","award":["G21S2861919"],"award-info":[{"award-number":["G21S2861919"]}]},{"name":"Princess Nourah bint Abdulrahman University Researchers Supporting Project","award":["NU\/RG\/SERC\/12\/40"],"award-info":[{"award-number":["NU\/RG\/SERC\/12\/40"]}]},{"name":"Princess Nourah bint Abdulrahman University Researchers Supporting Project","award":["PNURSP2023R440"],"award-info":[{"award-number":["PNURSP2023R440"]}]},{"name":"Princess Nourah bint Abdulrahman University Researchers Supporting Project","award":["G21S2861919"],"award-info":[{"award-number":["G21S2861919"]}]},{"name":"Ministry of SMEs and Startups (MSS, Korea)","award":["NU\/RG\/SERC\/12\/40"],"award-info":[{"award-number":["NU\/RG\/SERC\/12\/40"]}]},{"name":"Ministry of SMEs and Startups (MSS, Korea)","award":["PNURSP2023R440"],"award-info":[{"award-number":["PNURSP2023R440"]}]},{"name":"Ministry of SMEs and Startups (MSS, Korea)","award":["G21S2861919"],"award-info":[{"award-number":["G21S2861919"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Locomotion prediction for human welfare has gained tremendous interest in the past few years. Multimodal locomotion prediction is composed of small activities of daily living and an efficient approach to providing support for healthcare, but the complexities of motion signals along with video processing make it challenging for researchers in terms of achieving a good accuracy rate. The multimodal internet of things (IoT)-based locomotion classification has helped in solving these challenges. In this paper, we proposed a novel multimodal IoT-based locomotion classification technique using three benchmarked datasets. These datasets contain at least three types of data, such as data from physical motion, ambient, and vision-based sensors. The raw data has been filtered through different techniques for each sensor type. Then, the ambient and physical motion-based sensor data have been windowed, and a skeleton model has been retrieved from the vision-based data. Further, the features have been extracted and optimized using state-of-the-art methodologies. Lastly, experiments performed verified that the proposed locomotion classification system is superior when compared to other conventional approaches, particularly when considering multimodal data. The novel multimodal IoT-based locomotion classification system has achieved an accuracy rate of 87.67% and 86.71% over the HWU-USP and Opportunity++ datasets, respectively. The mean accuracy rate of 87.0% is higher than the traditional methods proposed in the literature.<\/jats:p>","DOI":"10.3390\/s23104716","type":"journal-article","created":{"date-parts":[[2023,5,12]],"date-time":"2023-05-12T10:49:51Z","timestamp":1683888591000},"page":"4716","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["A Multimodal IoT-Based Locomotion Classification System Using Features Engineering and Recursive Neural Network"],"prefix":"10.3390","volume":"23","author":[{"given":"Madiha","family":"Javeed","sequence":"first","affiliation":[{"name":"Department of Computer Science, Air University, Islamabad 44000, Pakistan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8361-6561","authenticated-orcid":false,"given":"Naif Al","family":"Mudawi","sequence":"additional","affiliation":[{"name":"Department of Computer Science, College of Computer Science and Information System, Najran University, Najran 55461, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bayan Ibrahimm","family":"Alabduallah","sequence":"additional","affiliation":[{"name":"Department of Information Systems, College of Computer and Information Sciences, Princess Nourah Bint Abdulrahman University, P.O. Box 84428, Riyadh 11671, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ahmad","family":"Jalal","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Air University, Islamabad 44000, Pakistan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0955-3421","authenticated-orcid":false,"given":"Wooseong","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of Computer Engineering, Gachon University, Seongnam 13120, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,5,12]]},"reference":[{"key":"ref_1","unstructured":"Ahmad, J., Nadeem, A., and Bobasu, S. (2019, January 6\u20137). Human Body Parts Estimation and Detection for Physical Sports Movements. Proceedings of the 2019 2nd International Conference on Communication, Computing and Digital Systems (C-CODE), Islamabad, Pakistan."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Pervaiz, M., and Ahmad, J. (2023, January 20\u201322). Artificial Neural Network for Human Object Interaction System Over Aerial Images. Proceedings of the 2023 4th International Conference on Advancements in Computational Sciences (ICACS), Lahore, Pakistan.","DOI":"10.1109\/ICACS55311.2023.10089722"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"6061","DOI":"10.1007\/s11042-019-08463-7","article-title":"Wearable sensors based human behavioral pattern recognition using statistical features and reweighted genetic algorithm","volume":"79","author":"Quaid","year":"2020","journal-title":"Multimed. Tools Appl."},{"key":"ref_4","unstructured":"and Azmat, U. (2022, January 21\u201322). Human Activity Recognition via Smartphone Embedded Sensor using Multi-Class SVM. Proceedings of the 2022 24th International Multitopic Conference (INMIC), Islamabad, Pakistan."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Ahmad, J., Batool, M., and Kim, K. (2020). Stochastic Recognition of Physical Activity and Healthcare Using Tri-Axial Inertial Wearable Sensors. Appl. Sci., 10.","DOI":"10.3390\/app10207122"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2797","DOI":"10.1007\/s10639-019-09892-5","article-title":"Students\u2019 behavior mining in e-learning environment using cognitive processes with information technologies","volume":"24","author":"Ahmad","year":"2019","journal-title":"Educ. Inf. Technol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"3491","DOI":"10.1109\/LRA.2021.3062562","article-title":"Real-Time Gait Phase Estimation for Robotic Hip Exoskeleton Control During Multimodal Locomotion","volume":"6","author":"Kang","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"6919","DOI":"10.1007\/s11042-019-08527-8","article-title":"WHITE STAG model: Wise human interaction tracking and estimation (WHITE) using spatio-temporal and angular-geometric (STAG) descriptors","volume":"79","author":"Mahmood","year":"2020","journal-title":"Multimed. Tools Appl."},{"key":"ref_9","unstructured":"Batool, M., Alotaibi, S.S., Alatiyyah, M.H., Alnowaiser, K., Aljuaid, H., Jalal, A., and Park, J. (2013). Depth Sensors-Based Action Recognition using a Modified K-Ary Entropy Classifier. IEEE Access."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"23964","DOI":"10.1109\/ACCESS.2022.3154775","article-title":"MS-DLD: Multi-Sensors Based Daily Locomotion Detection via Kinematic-Static Energy and Body-Specific HMMs","volume":"10","author":"Ghadi","year":"2022","journal-title":"IEEE Access"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"33250","DOI":"10.1109\/ACCESS.2020.2971552","article-title":"Daily Locomotion Recognition and Prediction: A Kinematic Data-Based Machine Learning Approach","volume":"8","author":"Figueiredo","year":"2020","journal-title":"IEEE Access"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Madiha, J., Shorfuzzaman, M., Alsufyani, N., Chelloug, S.A., Jalal, A., and Park, J. (2022). Physical human locomotion prediction using manifold regularization. PeerJ Comput. Sci., 8.","DOI":"10.7717\/peerj-cs.1105"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Wang, L., Ciliberto, M., Gjoreski, H., Lago, P., Murao, K., Okita, T., and Roggen, D. (2021, January 21\u201326). Locomotion and Transportation Mode Recognition from GPS and Radio Signals: Summary of SHL Challenge 2021. Proceedings of the Adjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers (UbiComp \u201821), Association for Computing Machinery, New York, NY, USA.","DOI":"10.1145\/3460418.3479373"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2033","DOI":"10.1016\/j.patrec.2012.12.014","article-title":"The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition","volume":"34","author":"Chavarriaga","year":"2013","journal-title":"Pattern Recognit. Lett."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Ord\u00f3\u00f1ez, F., and Roggen, D. (2016). Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition. Sensors, 16.","DOI":"10.3390\/s16010115"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1109\/MIC.2015.72","article-title":"Multimodal Wearable Sensing for Fine-Grained Activity Recognition in Healthcare","volume":"19","author":"De","year":"2015","journal-title":"IEEE Internet Comput."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Chung, S., Lim, J., Noh, K.J., Kim, G., and Jeong, H. (2019). Sensor Data Acquisition and Multimodal Sensor Fusion for Human Activity Recognition Using Deep Learning. Sensors, 19.","DOI":"10.3390\/s19071716"},{"key":"ref_18","unstructured":"Ahmad, J., and Kim, Y. (2014, January 26\u201329). Dense depth maps-based human pose tracking and recognition in dynamic scenes using ridge data. Proceedings of the 2014 11th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), Seoul, Republic of Korea."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Muneeb, M., Rustam, H., and Ahmad, J. (2023, January 20\u201322). Automate Appliances via Gestures Recognition for Elderly Living Assistance. Proceedings of the 2023 4th International Conference on Advancements in Computational Sciences (ICACS), Lahore, Pakistan.","DOI":"10.1109\/ICACS55311.2023.10089778"},{"key":"ref_20","unstructured":"Madiha, J., and Ahmad, J. (2021, January 9\u201310). Body-worn Hybrid-Sensors based Motion Patterns Detection via Bag-of-features and Fuzzy Logic Optimization. Proceedings of the 2021 International Conference on Innovative Computing (ICIC), Lahore, Pakistan."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"149","DOI":"10.32604\/iasc.2023.026051","article-title":"Student\u2019s health exercise recognition tool for E-learning education","volume":"35","author":"Shloul","year":"2023","journal-title":"IASC Intell. Autom. Soft Comput."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Gochoo, M., Akhter, I., Jalal, A., and Kim, K. (2021). Stochastic remote sensing event classification over adaptive posture estimation via multifused data and deep belief network. Remote Sens., 13.","DOI":"10.3390\/rs13050912"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Azmat, U., and Ahmad, J. (2021, January 21\u201322). Smartphone Inertial Sensors for Human Locomotion Activity Recognition based on Template Matching and Codebook Generation. Proceedings of the 2021 International Conference on Communication Technologies (ComTech), Rawalpindi, Pakistan.","DOI":"10.1109\/ComTech52583.2021.9616681"},{"key":"ref_24","unstructured":"Ahmad, J., Quaid, M.A.K., and Hasan, A.S. (2018, January 17\u201319). Wearable Sensor-Based Human Behavior Understanding and Recognition in Daily Life for Smart Environments. Proceedings of the 2018 International Conference on Frontiers of Information Technology (FIT), Islamabad, Pakistan."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1733","DOI":"10.1007\/s42835-019-00187-w","article-title":"A Wrist Worn Acceleration Based Human Motion Analysis and Classification for Ambient Smart Home System","volume":"14","author":"Ahmad","year":"2019","journal-title":"J. Electr. Eng. Technol."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Zhuo, S., Sherlock, L., Dobbie, G., Koh, Y.S., Russello, G., and Lottridge, D. (2020). Real-time Smartphone Activity Classification Using Inertial Sensors\u2014Recognition of Scrolling, Typing, and Watching Videos While Sitting or Walking. Sensors, 20.","DOI":"10.3390\/s20030655"},{"key":"ref_27","first-page":"28","article-title":"EEG Signal Classification using Linear Predictive Cepstral Coefficient Features","volume":"73","author":"Pazhanirajan","year":"2013","journal-title":"Int. J. Comput. Appl."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1245","DOI":"10.1007\/s10044-017-0605-z","article-title":"A New Descriptor for Image Matching Based on Bionic Principles","volume":"20","author":"Fausto","year":"2017","journal-title":"Pattern Anal. Appl."},{"key":"ref_29","unstructured":"Madiha, J., Jalal, A., and Kim, K. (2021, January 12\u201316). Wearable Sensors based Exertion Recognition using Statistical Features and Random Forest for Physical Healthcare Monitoring. Proceedings of the 2021 International Bhurban Conference on Applied Sciences and Technologies (IBCAST), Islamabad, Pakistan."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Sen, B., Hussain, S.A.I., Gupta, A.D., Gupta, M.K., Pimenov, D.Y., and Miko\u0142ajczyk, T. (2020). Application of Type-2 Fuzzy AHP-ARAS for Selecting Optimal WEDM Parameters. Metals, 11.","DOI":"10.3390\/met11010042"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"3025","DOI":"10.1109\/TCSVT.2020.3035722","article-title":"Recursive Neural Network for Video Deblurring","volume":"31","author":"Zhang","year":"2021","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Murad, A., and Pyun, J.-Y. (2017). Deep Recurrent Neural Networks for Human Activity Recognition. Sensors, 17.","DOI":"10.3390\/s17112556"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Ranieri, C.M., MacLeod, S., Dragone, M., Vargas, P.A., and Romero, R.F. (2021). Activity Recognition for Ambient Assisted Living with Videos, Inertial Units and Ambient Sensors. Sensors, 21.","DOI":"10.3390\/s21030768"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Ciliberto, M., Rey, V.F., Calatroni, A., Lukowicz, P., and Roggen, D. (2021). Opportunity++: A Multimodal Dataset for Video- and Wearable, Object and Ambient Sensors-based Human Activity Recognition. Front. Comput. Sci., 3.","DOI":"10.3389\/fcomp.2021.792065"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Akhter, I., Jalal, A., and Kim, K. (2021, January 12\u201316). Pose Estimation and Detection for Event Recognition using Sense-Aware Features and Adaboost Classifier. Proceedings of the 2021 International Bhurban Conference on Applied Sciences and Technologies (IBCAST), Islamabad, Pakistan.","DOI":"10.1109\/IBCAST51254.2021.9393293"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Javeed, M., and Jalal, A. (2023, January 20\u201322). Deep Activity Recognition based on Patterns Discovery for Healthcare Monitoring. Proceedings of the 2023 International Conference on Advancements in Computational Sciences (ICACS), Lahore, Pakistan.","DOI":"10.1109\/ICACS55311.2023.10089764"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"21465","DOI":"10.1007\/s11042-021-10687-5","article-title":"Automatic human posture estimation for sport activity recognition with robust body parts detection and entropy markov model","volume":"80","author":"Nadeem","year":"2021","journal-title":"Multimed. Tools Appl."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"116527","DOI":"10.1109\/ACCESS.2022.3214986","article-title":"Deep Human Motion Detection and Multi-Features Analysis for Smart Healthcare Learning Tools","volume":"10","author":"Hajjej","year":"2022","journal-title":"IEEE Access"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Memmesheimer, R., Theisen, N., and Paulus, D. (2020\u201324, January 24). Gimme Signals: Discriminative signal encoding for multimodal activity recognition. Proceedings of the 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA.","DOI":"10.1109\/IROS45743.2020.9341699"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Mart\u00ednez-Villase\u00f1or, L., Ponce, H., Brieva, J., Moya-Albor, E., N\u00fa\u00f1ez-Mart\u00ednez, J., and Pe\u00f1afort-Asturiano, C. (2019). UP-Fall Detection Dataset: A Multimodal Approach. Sensors, 19.","DOI":"10.3390\/s19091988"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Piechocki, R.J., Wang, X., and Bocus, M.J. (2023). Multimodal sensor fusion in the latent representation space. Sci. Rep., 13.","DOI":"10.1038\/s41598-022-24754-w"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1016\/j.promfg.2020.01.288","article-title":"Action Recognition in Manufacturing Assembly using Multimodal Sensor Fusion","volume":"39","author":"Tao","year":"2019","journal-title":"Procedia Manuf."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Gao, W., Zhang, L., Teng, Q., He, J., and Wu, H. (2021). DanHAR: Dual Attention Network for multimodal human activity recognition using wearable sensors. Appl. Soft Comput., 111.","DOI":"10.1016\/j.asoc.2021.107728"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Ahmad, J., Batool, M., and Kim, K. (2020). Sustainable Wearable System: Human Behavior Modeling for Life-Logging Activities Using K-Ary Tree Hashing Classifier. Sustainability, 12.","DOI":"10.3390\/su122410324"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/10\/4716\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:34:05Z","timestamp":1760124845000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/10\/4716"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,12]]},"references-count":44,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2023,5]]}},"alternative-id":["s23104716"],"URL":"https:\/\/doi.org\/10.3390\/s23104716","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,12]]}}}