{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,14]],"date-time":"2026-10-14T23:26:17Z","timestamp":1792020377008,"version":"4.4.0"},"reference-count":60,"publisher":"Public Library of Science (PLoS)","issue":"2","license":[{"start":{"date-parts":[[2026,2,23]],"date-time":"2026-02-23T00:00:00Z","timestamp":1771804800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"crossref","award":["RS-2023-00217471"],"award-info":[{"award-number":["RS-2023-00217471"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Princess Nourah bint Abdulrahman University Researchers","award":["PNURSP2026R508"],"award-info":[{"award-number":["PNURSP2026R508"]}]}],"content-domain":{"domain":["www.plosone.org"],"crossmark-restriction":false},"short-container-title":["PLoS One"],"abstract":"<jats:p>Innovative technologies for developing intelligent systems related to locomotion prediction learning are crucial in today\u2019s world. Human locomotion involves various complex concepts that must be addressed to enable accurate prediction through learning mechanisms. Our proposed system focuses on locomotion learning through vision RGB devices, ambient sensors-based signals, and physiological motions from biosensing devices. First, the data is acquired from five different scenarios-based datasets. Then, we pre-process the data to mitigate the noise from biosensors and extract body landmarks and key points from computer vision-based signals. The data is then segmented using a data windowing technique. Various features are extracted through multiple combinations of feature extraction methodologies, followed by feature reduction using optimization techniques. In contrast to existing systems, we employ both machine learning and deep learning classifiers for locomotion prediction, utilizing a modified body-specific sensor-based Hidden Markov Model and a deep Exponential Residual Neural Network, respectively. System ontology is also presented to elucidate the relationships among the data, concepts, and objects within the system. Experimental results indicate that our proposed biosensor-based system exhibits significant potential for effective locomotion prediction learning.<\/jats:p>","DOI":"10.1371\/journal.pone.0342793","type":"journal-article","created":{"date-parts":[[2026,2,23]],"date-time":"2026-02-23T18:29:17Z","timestamp":1771871357000},"page":"e0342793","update-policy":"https:\/\/doi.org\/10.1371\/journal.pone.corrections_policy","source":"Crossref","is-referenced-by-count":1,"title":["Deep locomotion prediction learning over biosensors, ambient sensors, and computer vision"],"prefix":"10.1371","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5093-7334","authenticated-orcid":true,"given":"Madiha","family":"Javeed","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ahmad","family":"Jalal","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dina Abdulaziz","family":"AlHammadi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2482-1869","authenticated-orcid":true,"given":"Bumshik","family":"Lee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,2,23]]},"reference":[{"issue":"15","key":"pone.0342793.ref001","doi-asserted-by":"crossref","first-page":"11729","DOI":"10.1007\/s00521-019-04658-z","article-title":"Learning inverse dynamics for human locomotion analysis","volume":"32","author":"P Zell","year":"2019","journal-title":"Neural Comput Applic"},{"key":"pone.0342793.ref002","doi-asserted-by":"crossref","unstructured":"Azmat U. Human Activity Recognition via Smartphone Embedded Sensor using Multi-Class SVM. 2022 24th International Multitopic Conference (INMIC), 2022. 1\u20137. doi: 10.1109\/inmic56986.2022.9972927","DOI":"10.1109\/INMIC56986.2022.9972927"},{"key":"pone.0342793.ref003","doi-asserted-by":"crossref","DOI":"10.7717\/peerj-cs.1105","article-title":"Physical human locomotion prediction using manifold regularization","volume":"8","author":"M Javeed","year":"2022","journal-title":"PeerJ Comput Sci"},{"key":"pone.0342793.ref004","doi-asserted-by":"crossref","unstructured":"Azmat U, Jalal A. Smartphone inertial sensors for human locomotion activity recognition based on template matching and codebook generation. 2021 International Conference on Communication Technologies (ComTech), 2021. 109\u201314. doi: 10.1109\/comtech52583.2021.9616681","DOI":"10.1109\/ComTech52583.2021.9616681"},{"key":"pone.0342793.ref005","doi-asserted-by":"crossref","first-page":"33250","DOI":"10.1109\/ACCESS.2020.2971552","article-title":"Daily locomotion recognition and prediction: A kinematic data-based machine learning approach","volume":"8","author":"J Figueiredo","year":"2020","journal-title":"IEEE Access"},{"issue":"5","key":"pone.0342793.ref006","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1109\/MIC.2015.72","article-title":"Multimodal wearable sensing for fine-grained activity recognition in healthcare","volume":"19","author":"D De","year":"2015","journal-title":"IEEE Internet Comput"},{"key":"pone.0342793.ref007","doi-asserted-by":"crossref","unstructured":"Jalal A, Kim Y. Dense depth maps-based human pose tracking and recognition in dynamic scenes using ridge data. 2014 11th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS), 2014. 119\u201324. doi: 10.1109\/avss.2014.6918654","DOI":"10.1109\/AVSS.2014.6918654"},{"issue":"1","key":"pone.0342793.ref008","doi-asserted-by":"crossref","first-page":"115","DOI":"10.3390\/s16010115","article-title":"Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition","volume":"16","author":"FJ Ord\u00f3\u00f1ez","year":"2016","journal-title":"Sensors (Basel)"},{"issue":"12","key":"pone.0342793.ref009","doi-asserted-by":"crossref","first-page":"2204","DOI":"10.3390\/mi14122204","article-title":"Biosensor-based multimodal deep human locomotion decoding via internet of healthcare things","volume":"14","author":"M Javeed","year":"2023","journal-title":"Micromachines (Basel)"},{"issue":"1","key":"pone.0342793.ref010","doi-asserted-by":"crossref","first-page":"4998","DOI":"10.1038\/s41598-022-26951-z","article-title":"Reshaping healthcare with wearable biosensors","volume":"13","author":"AA Smith","year":"2023","journal-title":"Sci Rep"},{"issue":"8","key":"pone.0342793.ref011","doi-asserted-by":"crossref","first-page":"3073","DOI":"10.1007\/s12652-017-0668-0","article-title":"Ontology-based sensor fusion activity recognition","volume":"11","author":"MHM Noor","year":"2018","journal-title":"J Ambient Intell Human Comput"},{"key":"pone.0342793.ref012","doi-asserted-by":"crossref","first-page":"105466","DOI":"10.1109\/ACCESS.2023.3317893","article-title":"Deep ontology-based human locomotor activity recognition system via multisensory devices","volume":"11","author":"M Javeed","year":"2023","journal-title":"IEEE Access"},{"issue":"5","key":"pone.0342793.ref013","doi-asserted-by":"crossref","first-page":"1193","DOI":"10.3390\/s19051193","article-title":"Towards semantic sensor data: An ontology approach","volume":"19","author":"J Liu","year":"2019","journal-title":"Sensors (Basel)"},{"issue":"5","key":"pone.0342793.ref014","doi-asserted-by":"crossref","first-page":"2517","DOI":"10.18280\/ts.410525","article-title":"Intelligent fine-grained daily living locomotion prediction based on skeleton modeling and CNN","volume":"41","author":"M Javeed","year":"2024","journal-title":"TS"},{"issue":"21","key":"pone.0342793.ref015","doi-asserted-by":"crossref","first-page":"8507","DOI":"10.3390\/s22218507","article-title":"A novel deep neural network method for HAR-based team training using body-worn inertial sensors","volume":"22","author":"Y-C Fan","year":"2022","journal-title":"Sensors (Basel)"},{"key":"pone.0342793.ref016","doi-asserted-by":"crossref","unstructured":"Batool M, Javeed M. Movement Disorders Detection in Parkinson\u2019s Patients using Hybrid Classifier. In: 2022 19th International Bhurban Conference on Applied Sciences and Technology (IBCAST), 2022. 213\u20138. doi: 10.1109\/ibcast54850.2022.9990423","DOI":"10.1109\/IBCAST54850.2022.9990423"},{"key":"pone.0342793.ref017","doi-asserted-by":"crossref","first-page":"68022","DOI":"10.1109\/ACCESS.2019.2917125","article-title":"SmartWall: Novel RFID-enabled ambient human activity recognition using machine learning for unobtrusive health monitoring","volume":"7","author":"GA Oguntala","year":"2019","journal-title":"IEEE Access"},{"issue":"1","key":"pone.0342793.ref018","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s00530-021-00875-6","article-title":"Towards a multimodal human activity dataset for healthcare","volume":"29","author":"M Hu","year":"2022","journal-title":"Multimedia Systems"},{"issue":"7","key":"pone.0342793.ref019","doi-asserted-by":"crossref","first-page":"1716","DOI":"10.3390\/s19071716","article-title":"Sensor data acquisition and multimodal sensor fusion for human activity recognition using deep learning","volume":"19","author":"S Chung","year":"2019","journal-title":"Sensors (Basel)"},{"key":"pone.0342793.ref020","doi-asserted-by":"crossref","first-page":"179028","DOI":"10.1109\/ACCESS.2020.3027979","article-title":"A deep learning approach for human activities recognition from multimodal sensing devices","volume":"8","author":"IK Ihianle","year":"2020","journal-title":"IEEE Access"},{"issue":"2","key":"pone.0342793.ref021","doi-asserted-by":"crossref","first-page":"1729","DOI":"10.1109\/LRA.2021.3059624","article-title":"Multi-GAT: A graphical attention-based hierarchical multimodal representation learning approach for human activity recognition","volume":"6","author":"MM Islam","year":"2021","journal-title":"IEEE Robot Autom Lett"},{"key":"pone.0342793.ref022","doi-asserted-by":"crossref","first-page":"116527","DOI":"10.1109\/ACCESS.2022.3214986","article-title":"Deep human motion detection and multi-features analysis for smart healthcare learning tools","volume":"10","author":"F Hajjej","year":"2022","journal-title":"IEEE Access"},{"key":"pone.0342793.ref023","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1109\/ACCESS.2021.3138614","article-title":"Efficient prediction of human motion for real-time robotics applications with physics-inspired neural networks","volume":"10","author":"A Antonucci","year":"2022","journal-title":"IEEE Access"},{"issue":"2","key":"pone.0342793.ref024","doi-asserted-by":"crossref","first-page":"2610","DOI":"10.1109\/LRA.2020.2972879","article-title":"Learning natural locomotion behaviors for humanoid robots using human bias","volume":"5","author":"C Yang","year":"2020","journal-title":"IEEE Robot Autom Lett"},{"issue":"1","key":"pone.0342793.ref025","doi-asserted-by":"crossref","first-page":"149","DOI":"10.32604\/iasc.2023.026051","article-title":"Student\u2019s Health exercise recognition tool for e-learning education","volume":"35","author":"T al Shloul","year":"2023","journal-title":"Intelligent Automation & Soft Computing"},{"key":"pone.0342793.ref026","author":"M Ciliberto","year":"2021"},{"key":"pone.0342793.ref027","unstructured":"Torre FD, Hodgins JK, Bargteil AW, Martin X, Macey J, Collado AT, et al. Guide to the Carnegie Mellon University Multimodal Activity (CMU-MMAC) Database. 2008."},{"key":"pone.0342793.ref028","doi-asserted-by":"crossref","unstructured":"Ofli F, Chaudhry R, Kurillo G, Vidal R, Bajcsy R. Berkeley MHAD: A comprehensive Multimodal Human Action Database. 2013 IEEE Workshop on Applications of Computer Vision (WACV), 2013. 53\u201360. doi: 10.1109\/wacv.2013.6474999","DOI":"10.1109\/WACV.2013.6474999"},{"issue":"3","key":"pone.0342793.ref029","doi-asserted-by":"crossref","first-page":"768","DOI":"10.3390\/s21030768","article-title":"Activity recognition for ambient assisted living with videos, inertial units and ambient sensors","volume":"21","author":"CM Ranieri","year":"2021","journal-title":"Sensors (Basel)"},{"issue":"15","key":"pone.0342793.ref030","doi-asserted-by":"crossref","first-page":"4083","DOI":"10.3390\/s20154083","article-title":"LARa: Creating a dataset for human activity recognition in logistics using semantic attributes","volume":"20","author":"F Niemann","year":"2020","journal-title":"Sensors (Basel)"},{"key":"pone.0342793.ref031","doi-asserted-by":"crossref","unstructured":"Javeed M, Jalal A. Body-worn Hybrid-Sensors based Motion Patterns Detection via Bag-of-features and Fuzzy Logic Optimization. 2021 International Conference on Innovative Computing (ICIC), 2021. 1\u20137. doi: 10.1109\/icic53490.2021.9692924","DOI":"10.1109\/ICIC53490.2021.9692924"},{"key":"pone.0342793.ref032","doi-asserted-by":"crossref","first-page":"23964","DOI":"10.1109\/ACCESS.2022.3154775","article-title":"MS-DLD: Multi-sensors based daily locomotion detection via kinematic-static energy and body-specific HMMs","volume":"10","author":"YY Ghadi","year":"2022","journal-title":"IEEE Access"},{"issue":"15","key":"pone.0342793.ref033","doi-asserted-by":"crossref","first-page":"3030","DOI":"10.3390\/nu14153030","article-title":"Phase angle, a cornerstone of outcome in head and neck cancer","volume":"14","author":"D Sat-Mu\u00f1oz","year":"2022","journal-title":"Nutrients"},{"issue":"9","key":"pone.0342793.ref034","first-page":"7975","article-title":"Design and implementation of Butterworth filter","volume":"9","author":"M Shouran","year":"2020","journal-title":"International Journal of Innovative Research in Science, Engineering and Technology"},{"key":"pone.0342793.ref035","doi-asserted-by":"crossref","unstructured":"Akhter I, Hafeez S. Human Body 3D Reconstruction and Gait Analysis via Features Mining Framework. 2022 19th International Bhurban Conference on Applied Sciences and Technology (IBCAST), 2022. 189\u201394. doi: 10.1109\/ibcast54850.2022.9990213","DOI":"10.1109\/IBCAST54850.2022.9990213"},{"issue":"10","key":"pone.0342793.ref036","doi-asserted-by":"crossref","first-page":"1667","DOI":"10.3390\/rs12101667","article-title":"Object detection and image segmentation with deep learning on earth observation data: A Review-Part I: Evolution and Recent Trends","volume":"12","author":"T Hoeser","year":"2020","journal-title":"Remote Sensing"},{"issue":"7","key":"pone.0342793.ref037","doi-asserted-by":"crossref","first-page":"5441","DOI":"10.1109\/TIE.2018.2868296","article-title":"Multisynchrosqueezing Transform","volume":"66","author":"G Yu","year":"2019","journal-title":"IEEE Trans Ind Electron"},{"issue":"4","key":"pone.0342793.ref038","doi-asserted-by":"crossref","first-page":"4167","DOI":"10.1109\/TIE.2021.3075871","article-title":"SuperGraph: Spatial-temporal graph-based feature extraction for rotating machinery diagnosis","volume":"69","author":"C Yang","year":"2022","journal-title":"IEEE Trans Ind Electron"},{"key":"pone.0342793.ref039","doi-asserted-by":"crossref","first-page":"108569","DOI":"10.1016\/j.measurement.2020.108569","article-title":"Synchrosqueezing extracting transform and its application in bearing fault diagnosis under non-stationary conditions","volume":"173","author":"Q Liu","year":"2021","journal-title":"Measurement"},{"issue":"1","key":"pone.0342793.ref040","article-title":"Graph neural networks for an accurate and interpretable prediction of the properties of polycrystalline materials","volume":"7","author":"M Dai","year":"2021","journal-title":"npj Comput Mater"},{"issue":"5","key":"pone.0342793.ref041","doi-asserted-by":"crossref","first-page":"6086","DOI":"10.1109\/TPAMI.2022.3203157","article-title":"Parameterized hamiltonian learning with quantum circuit","volume":"45","author":"J Shi","year":"2023","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"issue":"2","key":"pone.0342793.ref042","doi-asserted-by":"crossref","first-page":"1771","DOI":"10.1007\/s11063-019-10185-8","article-title":"Daily activity feature selection in smart homes based on pearson correlation coefficient","volume":"51","author":"Y Liu","year":"2020","journal-title":"Neural Process Lett"},{"issue":"9","key":"pone.0342793.ref043","doi-asserted-by":"crossref","first-page":"3332","DOI":"10.3390\/s22093332","article-title":"A Comparison and evaluation of stereo matching on active stereo images","volume":"22","author":"M Jang","year":"2022","journal-title":"Sensors (Basel)"},{"key":"pone.0342793.ref044","doi-asserted-by":"crossref","unstructured":"Javeed M, Jalal A, Kim K. Wearable Sensors based Exertion Recognition using Statistical Features and Random Forest for Physical Healthcare Monitoring. 2021 International Bhurban Conference on Applied Sciences and Technologies (IBCAST), 2021. 512\u20137. doi: 10.1109\/ibcast51254.2021.9393014","DOI":"10.1109\/IBCAST51254.2021.9393014"},{"issue":"7","key":"pone.0342793.ref045","doi-asserted-by":"crossref","first-page":"2056","DOI":"10.3390\/s20072056","article-title":"Optimizing sensor ontology alignment through compact co-firefly algorithm","volume":"20","author":"X Xue","year":"2020","journal-title":"Sensors (Basel)"},{"issue":"5","key":"pone.0342793.ref046","doi-asserted-by":"crossref","first-page":"1193","DOI":"10.3390\/s19051193","article-title":"Towards semantic sensor data: An ontology approach","volume":"19","author":"J Liu","year":"2019","journal-title":"Sensors (Basel)"},{"issue":"3","key":"pone.0342793.ref047","doi-asserted-by":"crossref","first-page":"1390","DOI":"10.3390\/s23031390","article-title":"Human Activity Recognition with an HMM-Based Generative Model","volume":"23","author":"N Manouchehri","year":"2023","journal-title":"Sensors (Basel)"},{"issue":"24","key":"pone.0342793.ref048","doi-asserted-by":"crossref","first-page":"12571","DOI":"10.3390\/app122412571","article-title":"Predicting the porosity in selective laser melting parts using hybrid regression convolutional neural network","volume":"12","author":"NMH Alamri","year":"2022","journal-title":"Applied Sciences"},{"key":"pone.0342793.ref049","doi-asserted-by":"crossref","unstructured":"Azmat U, Jalal A, Javeed M. Multi-sensors Fused IoT-based Home Surveillance via Bag of Visual and Motion Features. 2023 International Conference on Communication, Computing and Digital Systems (C-CODE), 2023. 1\u20136. doi: 10.1109\/c-code58145.2023.10139889","DOI":"10.1109\/C-CODE58145.2023.10139889"},{"key":"pone.0342793.ref050","doi-asserted-by":"crossref","unstructured":"Akhter I, Javeed M, Jalal A. Deep Skeleton Modeling and Hybrid Hand-crafted Cues over Physical Exercises. 2023 International Conference on Communication, Computing and Digital Systems (C-CODE), 2023. doi: 10.1109\/c-code58145.2023.10139863","DOI":"10.1109\/C-CODE58145.2023.10139863"},{"issue":"23","key":"pone.0342793.ref051","doi-asserted-by":"crossref","first-page":"11403","DOI":"10.1109\/JSEN.2019.2934678","article-title":"Autonomous human activity classification from wearable multi-modal sensors","volume":"19","author":"Y Lu","year":"2019","journal-title":"IEEE Sensors J"},{"issue":"6","key":"pone.0342793.ref052","doi-asserted-by":"crossref","first-page":"2801","DOI":"10.1007\/s42835-020-00554-y","article-title":"Telemonitoring of daily activity using accelerometer and gyroscope in smart home environments","volume":"15","author":"M Batool","year":"2020","journal-title":"J Electr Eng Technol"},{"issue":"3","key":"pone.0342793.ref053","doi-asserted-by":"crossref","first-page":"4935","DOI":"10.32604\/cmc.2022.023841","article-title":"Sensors-based ambient assistant living via e-monitoring technology","volume":"73","author":"S Hafeez","year":"2022","journal-title":"Computers, Materials Continua"},{"key":"pone.0342793.ref054","doi-asserted-by":"crossref","first-page":"29233","DOI":"10.1109\/ACCESS.2022.3157605","article-title":"Human Motion Enhancement via Tobit Kalman Filter-Assisted Autoencoder","volume":"10","author":"N Lannan","year":"2022","journal-title":"IEEE Access"},{"issue":"1","key":"pone.0342793.ref055","doi-asserted-by":"crossref","first-page":"726","DOI":"10.1038\/s41597-022-01841-1","article-title":"Construction motion data library: an integrated motion dataset for on-site activity recognition","volume":"9","author":"Y Tian","year":"2022","journal-title":"Sci Data"},{"key":"pone.0342793.ref056","doi-asserted-by":"crossref","unstructured":"Lannan N, Zhou L, Fan G. A Multiview Depth-based Motion Capture Benchmark Dataset for Human Motion Denoising and Enhancement Research. 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2022. 426\u201335. doi: 10.1109\/cvprw56347.2022.00058","DOI":"10.1109\/CVPRW56347.2022.00058"},{"key":"pone.0342793.ref057","unstructured":"L\u00fcdtke S, Rueda FM, Ahmed W, Fink GA, Kirste T. Human activity recognition using attribute-based neural networks and context information. doi: 10.48550\/arXiv.2111.04564"},{"key":"pone.0342793.ref058","doi-asserted-by":"crossref","unstructured":"Awasthi S, Rueda FM, Fink GA. Video-based Pose-Estimation Data as Source for Transfer Learning in Human Activity Recognition. 2022 26th International Conference on Pattern Recognition (ICPR), 2022. 4514\u201321. doi: 10.1109\/icpr56361.2022.9956405","DOI":"10.1109\/ICPR56361.2022.9956405"},{"issue":"9","key":"pone.0342793.ref059","article-title":"Using Wearable sensors for human activity recognition in logistics: A comparison of different feature sets and machine learning algorithms","volume":"11","author":"AS Syed","year":"2020","journal-title":"IJACSA"},{"issue":"10","key":"pone.0342793.ref060","doi-asserted-by":"crossref","first-page":"4716","DOI":"10.3390\/s23104716","article-title":"A multimodal IoT-based locomotion classification system using features engineering and recursive neural network","volume":"23","author":"M Javeed","year":"2023","journal-title":"Sensors (Basel)"}],"container-title":["PLOS One"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0342793","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,23]],"date-time":"2026-02-23T18:29:30Z","timestamp":1771871370000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pone.0342793"}},"subtitle":[],"editor":[{"given":"Andrea","family":"Tigrini","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,2,23]]},"references-count":60,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2026,2,23]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pone.0342793","relation":{},"ISSN":["1932-6203"],"issn-type":[{"value":"1932-6203","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,23]]}}}