{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,19]],"date-time":"2026-05-19T22:06:34Z","timestamp":1779228394139,"version":"3.51.4"},"reference-count":47,"publisher":"MDPI AG","issue":"22","license":[{"start":{"date-parts":[[2023,11,10]],"date-time":"2023-11-10T00:00:00Z","timestamp":1699574400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Science and Technology Major Project of Hubei Province of China","award":["2021AAA007"],"award-info":[{"award-number":["2021AAA007"]}]},{"name":"Science and Technology Major Project of Hubei Province of China","award":["BSQD2020009"],"award-info":[{"award-number":["BSQD2020009"]}]},{"name":"Science and Technology Major Project of Hubei Province of China","award":["KFJJ-2022017"],"award-info":[{"award-number":["KFJJ-2022017"]}]},{"name":"Research Start-up Fund from Hubei University of Technology","award":["2021AAA007"],"award-info":[{"award-number":["2021AAA007"]}]},{"name":"Research Start-up Fund from Hubei University of Technology","award":["BSQD2020009"],"award-info":[{"award-number":["BSQD2020009"]}]},{"name":"Research Start-up Fund from Hubei University of Technology","award":["KFJJ-2022017"],"award-info":[{"award-number":["KFJJ-2022017"]}]},{"name":"open fund from Hubei Modern Manufacturing Quality Engineering Key Laboratory","award":["2021AAA007"],"award-info":[{"award-number":["2021AAA007"]}]},{"name":"open fund from Hubei Modern Manufacturing Quality Engineering Key Laboratory","award":["BSQD2020009"],"award-info":[{"award-number":["BSQD2020009"]}]},{"name":"open fund from Hubei Modern Manufacturing Quality Engineering Key Laboratory","award":["KFJJ-2022017"],"award-info":[{"award-number":["KFJJ-2022017"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Global aging leads to a surge in neurological diseases. Quantitative gait analysis for the early detection of neurological diseases can effectively reduce the impact of the diseases. Recently, extensive research has focused on gait-abnormality-recognition algorithms using a single type of portable sensor. However, these studies are limited by the sensor\u2019s type and the task specificity, constraining the widespread application of quantitative gait recognition. In this study, we propose a multimodal gait-abnormality-recognition framework based on a Convolutional Neural Network-Bidirectional Long Short-Term Memory (CNN-BiLSTM) network. The as-established framework effectively addresses the challenges arising from smooth data interference and lengthy time series by employing an adaptive sliding window technique. Then, we convert the time series into time\u2013frequency plots to capture the characteristic variations in different abnormality gaits and achieve a unified representation of the multiple data types. This makes our signal processing method adaptable to several types of sensors. Additionally, we use a pre-trained Deep Convolutional Neural Network (DCNN) for feature extraction, and the consequently established CNN-BiLSTM network can achieve high-accuracy recognition by fusing and classifying the multi-sensor input data. To validate the proposed method, we conducted diversified experiments to recognize the gait abnormalities caused by different neuropathic diseases, such as amyotrophic lateral sclerosis (ALS), Parkinson\u2019s disease (PD), and Huntington\u2019s disease (HD). In the PDgait dataset, the framework achieved an accuracy of 98.89% in the classification of Parkinson\u2019s disease severity, surpassing DCLSTM\u2019s 96.71%. Moreover, the recognition accuracy of ALS, PD, and HD on the PDgait dataset was 100%, 96.97%, and 95.43% respectively, surpassing the majority of previously reported methods. These experimental results strongly demonstrate the potential of the proposed multimodal framework for gait abnormality identification. Due to the advantages of the framework, such as its suitability for different types of sensors and fewer training parameters, it is more suitable for gait monitoring in daily life and the customization of medical rehabilitation schedules, which will help more patients alleviate the harm caused by their diseases.<\/jats:p>","DOI":"10.3390\/s23229101","type":"journal-article","created":{"date-parts":[[2023,11,13]],"date-time":"2023-11-13T02:46:47Z","timestamp":1699843607000},"page":"9101","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":27,"title":["Multimodal Gait Abnormality Recognition Using a Convolutional Neural Network\u2013Bidirectional Long Short-Term Memory (CNN-BiLSTM) Network Based on Multi-Sensor Data Fusion"],"prefix":"10.3390","volume":"23","author":[{"given":"Jing","family":"Li","sequence":"first","affiliation":[{"name":"School of Mechanical Engineering and Hubei Modern Manufacturing Quality Engineering Key Laboratory, Hubei University of Technology, Wuhan 430068, China"},{"name":"School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Weisheng","family":"Liang","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering and Hubei Modern Manufacturing Quality Engineering Key Laboratory, Hubei University of Technology, Wuhan 430068, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiyan","family":"Yin","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering and Hubei Modern Manufacturing Quality Engineering Key Laboratory, Hubei University of Technology, Wuhan 430068, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jun","family":"Li","sequence":"additional","affiliation":[{"name":"Detroit Green Technology Institute, Hubei University of Technology, Wuhan 430068, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Weizheng","family":"Guan","sequence":"additional","affiliation":[{"name":"Detroit Green Technology Institute, Hubei University of Technology, Wuhan 430068, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,11,10]]},"reference":[{"key":"ref_1","unstructured":"World Health Organization (2020). Decade of Healthy Ageing: Baseline Report."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"e551","DOI":"10.1016\/S2468-2667(20)30190-0","article-title":"The burden of neurological diseases in Europe: An analysis for the Global Burden of Disease Study 2017","volume":"5","author":"Deuschl","year":"2020","journal-title":"Lancet Public Health"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1016\/j.humov.2016.03.010","article-title":"Gait variability in people with neurological disorders: A systematic review and meta-analysis","volume":"47","author":"Moon","year":"2016","journal-title":"Hum. Mov. Sci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3233\/ADR-170001","article-title":"Whole-day gait monitoring in patients with Alzheimer\u2019s disease: A relationship between attention and gait cycle","volume":"1","author":"Higuma","year":"2017","journal-title":"J. Alzheimer\u2019s Dis. Rep."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Sarajchi, M., and Sirlantzis, K. (2023). Design and Control of a Single-Leg Exoskeleton with Gravity Compensation for Children with Unilateral Cerebral Palsy. Sensors, 23.","DOI":"10.3390\/s23136103"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.medengphy.2017.12.006","article-title":"Automatic recognition of gait patterns in human motor disorders using machine learning: A review","volume":"53","author":"Figueiredo","year":"2018","journal-title":"Med. Eng. Phys."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1521","DOI":"10.1109\/JBHI.2016.2608720","article-title":"Toward pervasive gait analysis with wearable sensors: A systematic review","volume":"20","author":"Chen","year":"2016","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"3362","DOI":"10.3390\/s140203362","article-title":"Gait analysis methods: An overview of wearable and non-wearable systems, highlighting clinical applications","volume":"14","year":"2014","journal-title":"Sensors"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1109\/JBHI.2021.3092875","article-title":"Human gait analysis in neurodegenerative diseases: A review","volume":"26","author":"Cicirelli","year":"2021","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"163180","DOI":"10.1109\/ACCESS.2019.2950254","article-title":"Abnormal gait recognition algorithm based on LSTM-CNN fusion network","volume":"7","author":"Gao","year":"2019","journal-title":"IEEE Access"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Jani, D., Varadarajan, V., Parmar, R., Bohara, M.H., Garg, D., Ganatra, A., and Kotecha, K. (2022). An efficient gait abnormality detection method based on classification. J. Sens. Actuator Netw., 11.","DOI":"10.3390\/jsan11030031"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1838","DOI":"10.1109\/JSEN.2020.3018262","article-title":"Gait spatiotemporal signal analysis for Parkinson\u2019s disease detection and severity rating","volume":"21","author":"Alharthi","year":"2020","journal-title":"IEEE Sens. J."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Yurdakul, O.C., Subathra, M., and George, S.T. (2020). detection of parkinson\u2019s disease from gait using neighborhood representation local binary patterns. Biomed. Signal Process. Control, 62.","DOI":"10.1016\/j.bspc.2020.102070"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"9439","DOI":"10.1109\/TCYB.2021.3056104","article-title":"Multimodal gait recognition for neurodegenerative diseases","volume":"52","author":"Zhao","year":"2021","journal-title":"IEEE Trans. Cybern."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s13673-019-0194-5","article-title":"Multi-sensor fusion based on multiple classifier systems for human activity identification","volume":"9","author":"Nweke","year":"2019","journal-title":"Hum.-Cent. Comput. Inf. Sci."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"e215","DOI":"10.1161\/01.CIR.101.23.e215","article-title":"PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals","volume":"101","author":"Goldberger","year":"2000","journal-title":"Circulation"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1016\/0021-9290(94)00074-E","article-title":"Footswitch system for measurement of the temporal parameters of gait","volume":"28","author":"Hausdorff","year":"1995","journal-title":"J. Biomech."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"262","DOI":"10.1152\/jappl.1997.82.1.262","article-title":"Altered fractal dynamics of gait: Reduced stride-interval correlations with aging and Huntington\u2019s disease","volume":"82","author":"Hausdorff","year":"1997","journal-title":"J. Appl. Physiol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1016\/S0004-9514(14)60487-6","article-title":"Observation and analysis of hemiplegic gait: Swing phase","volume":"39","author":"Moore","year":"1993","journal-title":"Aust. J. Physiother."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"280","DOI":"10.1016\/j.gaitpost.2012.07.018","article-title":"Age and Parkinson\u2019s disease related kinematic alterations during multi-directional gait initiation","volume":"37","author":"Vallabhajosula","year":"2013","journal-title":"Gait Posture"},{"key":"ref_21","first-page":"635","article-title":"Gait disturbances in patients with stroke","volume":"6","author":"Balaban","year":"2014","journal-title":"Phys. Med. Rehabil."},{"key":"ref_22","unstructured":"Berker, N., and Yal\u00e7\u0131n, S. (2005). The HELP Guide to Cerebral Palsy, Global-HELP Publication."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"27","DOI":"10.4103\/2395-4264.188154","article-title":"Coupling effect of neuromuscular electrical stimulation on glutei and transcutaneous electrical nerve stimulation on hip adductors in scissoring gait","volume":"2","author":"Shaik","year":"2016","journal-title":"Indian J. Cereb. Palsy"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Chen, M., Huang, B., and Xu, Y. (2007, January 8\u201311). Human abnormal gait modeling via hidden Markov model. Proceedings of the 2007 International Conference on Information Acquisition, Seogwipo, Republic of Korea.","DOI":"10.1109\/ICIA.2007.4295787"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1016\/j.medengphy.2014.11.008","article-title":"Activity classification in persons with stroke based on frequency features","volume":"37","author":"Laudanski","year":"2015","journal-title":"Med. Eng. Phys."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1109\/JBHI.2016.2633287","article-title":"A deep learning approach to on-node sensor data analytics for mobile or wearable devices","volume":"21","author":"Ravi","year":"2016","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"677","DOI":"10.1007\/s10916-010-9535-7","article-title":"Automated screening of arrhythmia using wavelet based machine learning techniques","volume":"36","author":"Martis","year":"2012","journal-title":"J. Med. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1007\/s13246-015-0333-x","article-title":"Feature extraction and classification for EEG signals using wavelet transform and machine learning techniques","volume":"38","author":"Amin","year":"2015","journal-title":"Australas. Phys. Eng. Sci. Med."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Huang, Z., Ma, Y., Wang, R., Li, W., and Dai, Y. (2023). A Model for EEG-Based Emotion Recognition: CNN-Bi-LSTM with Attention Mechanism. Electronics, 12.","DOI":"10.3390\/electronics12143188"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Donahue, J., Anne Hendricks, L., Guadarrama, S., Rohrbach, M., Venugopalan, S., Saenko, K., and Darrell, T. (2015, January 7\u201312). Long-term recurrent convolutional networks for visual recognition and description. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298878"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Sun, P., Wang, J., and Dong, Z. (2023). CNN\u2013LSTM Neural Network for Identification of Pre-Cooked Pasta Products in Different Physical States Using Infrared Spectroscopy. Sensors, 23.","DOI":"10.3390\/s23104815"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Zheng, Y., and Blasch, E. (2023). Facial Micro-Expression Recognition Enhanced by Score Fusion and a Hybrid Model from Convolutional LSTM and Vision Transformer. Sensors, 23.","DOI":"10.3390\/s23125650"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Shi, L.F., Liu, Z.Y., Zhou, K.J., Shi, Y., and Jing, X. (2023). Novel Deep Learning Network for Gait Recognition Using Multimodal Inertial Sensors. Sensors, 23.","DOI":"10.3390\/s23020849"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"9903518","DOI":"10.1155\/2021\/9903518","article-title":"Multivariate cnn-lstm model for multiple parallel financial time-series prediction","volume":"2021","author":"Widiputra","year":"2021","journal-title":"Complexity"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"437","DOI":"10.3390\/app10020437","article-title":"Portfolio optimization-based stock prediction using long-short term memory network in quantitative trading","volume":"10","author":"Tadesse","year":"2020","journal-title":"Appl. Sci."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Lu, Y., Wang, H., Qi, Y., and Xi, H. (2021). Evaluation of classification performance in human lower limb jump phases of signal correlation information and LSTM models. Biomed. Signal Process. Control, 64.","DOI":"10.1016\/j.bspc.2020.102279"},{"key":"ref_37","unstructured":"Sabour, S., Frosst, N., and Hinton, G.E. (2017). Dynamic routing between capsules. Adv. Neural Inf. Process. Syst., 30."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1016\/j.jbi.2016.01.014","article-title":"A Q-backpropagated time delay neural network for diagnosing severity of gait disturbances in Parkinson\u2019s disease","volume":"60","author":"Jane","year":"2016","journal-title":"J. Biomed. Inform."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.neucom.2018.03.032","article-title":"A hybrid spatio-temporal model for detection and severity rating of Parkinson\u2019s disease from gait data","volume":"315","author":"Zhao","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1016\/j.knosys.2018.01.004","article-title":"Dual channel LSTM based multi-feature extraction in gait for diagnosis of Neurodegenerative diseases","volume":"145","author":"Zhao","year":"2018","journal-title":"Knowl.-Based Syst."},{"key":"ref_41","first-page":"23","article-title":"Detection of movement disorders using multi SVM","volume":"13","author":"Athisakthi","year":"2013","journal-title":"Glob. J. Comput. Sci. Technol."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"246","DOI":"10.1016\/j.ins.2015.04.047","article-title":"Classification of neurodegenerative diseases using gait dynamics via deterministic learning","volume":"317","author":"Zeng","year":"2015","journal-title":"Inf. Sci."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"7359","DOI":"10.1016\/j.eswa.2010.12.091","article-title":"Introduction and application of an automatic gait recognition method to diagnose movement disorders that arose of similar causes","volume":"38","author":"Banaie","year":"2011","journal-title":"Expert Syst. Appl."},{"key":"ref_44","unstructured":"S\u00e1nchez-Delacruz, E., Acosta-Escalante, F., Wister, M.A., Hern\u00e1ndez-Nolasco, J.A., Pancardo, P., and M\u00e9ndez-Castillo, J.J. (2014). Proceedings of the Ubiquitous Computing and Ambient Intelligence. Personalisation and User Adapted Services: 8th International Conference, UCAmI 2014, Belfast, UK, 2\u20135 December 2014, Springer International Publishing. Proceedings 8."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Lin, C.W., Wen, T.C., and Setiawan, F. (2020). Evaluation of Vertical Ground Reaction Forces Pattern Visualization in Neurodegenerative Diseases Identification Using Deep Learning and Recurrence Plot Image Feature Extraction. Sensors, 20.","DOI":"10.3390\/s20143857"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Erda\u015f, \u00c7.B., S\u00fcmer, E., and Kibaro\u011flu, S. (2021). Neurodegenerative disease detection and severity prediction using deep learning approaches. Biomed. Signal Process. Control, 70.","DOI":"10.1016\/j.bspc.2021.103069"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"20034","DOI":"10.1007\/s10489-023-04557-w","article-title":"NDDNet: A deep learning model for predicting neurodegenerative diseases from gait pattern","volume":"53","author":"Faisal","year":"2023","journal-title":"Appl. Intell."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/22\/9101\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:21:12Z","timestamp":1760131272000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/22\/9101"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,10]]},"references-count":47,"journal-issue":{"issue":"22","published-online":{"date-parts":[[2023,11]]}},"alternative-id":["s23229101"],"URL":"https:\/\/doi.org\/10.3390\/s23229101","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,10]]}}}