{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,22]],"date-time":"2026-04-22T09:37:32Z","timestamp":1776850652672,"version":"3.51.2"},"reference-count":46,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2021,9,16]],"date-time":"2021-09-16T00:00:00Z","timestamp":1631750400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100006111","name":"Minist\u00e9rio da Ci\u00eancia, Tecnologia e Ensino Superior","doi-asserted-by":"publisher","award":["UIDB\/50008\/2020"],"award-info":[{"award-number":["UIDB\/50008\/2020"]}],"id":[{"id":"10.13039\/501100006111","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Human motion analysis provides useful information for the diagnosis and recovery assessment of people suffering from pathologies, such as those affecting the way of walking, i.e., gait. With recent developments in deep learning, state-of-the-art performance can now be achieved using a single 2D-RGB-camera-based gait analysis system, offering an objective assessment of gait-related pathologies. Such systems provide a valuable complement\/alternative to the current standard practice of subjective assessment. Most 2D-RGB-camera-based gait analysis approaches rely on compact gait representations, such as the gait energy image, which summarize the characteristics of a walking sequence into one single image. However, such compact representations do not fully capture the temporal information and dependencies between successive gait movements. This limitation is addressed by proposing a spatiotemporal deep learning approach that uses a selection of key frames to represent a gait cycle. Convolutional and recurrent deep neural networks were combined, processing each gait cycle as a collection of silhouette key frames, allowing the system to learn temporal patterns among the spatial features extracted at individual time instants. Trained with gait sequences from the GAIT-IT dataset, the proposed system is able to improve gait pathology classification accuracy, outperforming state-of-the-art solutions and achieving improved generalization on cross-dataset tests.<\/jats:p>","DOI":"10.3390\/s21186202","type":"journal-article","created":{"date-parts":[[2021,9,22]],"date-time":"2021-09-22T03:47:35Z","timestamp":1632282455000},"page":"6202","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":37,"title":["A Spatiotemporal Deep Learning Approach for Automatic Pathological Gait Classification"],"prefix":"10.3390","volume":"21","author":[{"given":"Pedro","family":"Albuquerque","sequence":"first","affiliation":[{"name":"Instituto de Telecomunica\u00e7\u00f5es, Instituto Superior T\u00e9cnico, Universidade de Lisboa, Av. Rovisco Pais 1, 1049-001 Lisboa, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5534-4351","authenticated-orcid":false,"given":"Tanmay Tulsidas","family":"Verlekar","sequence":"additional","affiliation":[{"name":"Department of CSIS and APPCAIR, BITS Pilani, K K Birla, Goa Campus, Goa 403726, India"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6525-9572","authenticated-orcid":false,"given":"Paulo Lobato","family":"Correia","sequence":"additional","affiliation":[{"name":"Instituto de Telecomunica\u00e7\u00f5es, Instituto Superior T\u00e9cnico, Universidade de Lisboa, Av. Rovisco Pais 1, 1049-001 Lisboa, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9738-639X","authenticated-orcid":false,"given":"Lu\u00eds Ducla","family":"Soares","sequence":"additional","affiliation":[{"name":"Instituto de Telecomunica\u00e7\u00f5es, Instituto Universit\u00e1rio de Lisboa (ISCTE-IUL), Av. das For\u00e7as Armadas, 1649-026 Lisboa, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2021,9,16]]},"reference":[{"key":"ref_1","unstructured":"Gafurov, D. (2007, January 19\u201321). A survey of biometric gait recognition: Approaches, security and challenges. Proceedings of the Annual Norwegian Computer Science Conference Norway, Oslo, Norway."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Karampelas, P., and Bourlai, T. (2018). A survey of using biometrics for smart visual surveillance: Gait recognition. Surveillance in Action: Technologies for Civilian, Military and Cyber Surveillance, Springer International Publishing.","DOI":"10.1007\/978-3-319-68533-5"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Verlekar, T., Correia, P., and Soares, L. (September, January 28). Gait recognition using normalized shadows. Proceedings of the 2017 25th European Signal Processing Conference (EUSIPCO), Kos, Greece.","DOI":"10.23919\/EUSIPCO.2017.8081345"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"3362","DOI":"10.3390\/s140203362","article-title":"Gait analysis methods: An overview of wearable and non-wearable systems, highlighting clinical applications","volume":"14","year":"2014","journal-title":"Sensors"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Kessler, E., Malladi, V.V.S., and Tarazaga, P.A. (2019). Vibration-based gait analysis via instrumented buildings. Int. J. Distrib. Sens. Netw., 15.","DOI":"10.1177\/1550147719881608"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Chaparro-Rico, B.D.M., Cafolla, D., Tortola, P., and Galardi, G. (2020). Assessing Stiffness, Joint Torque and ROM for Paretic and Non-Paretic Lower Limbs during the Subacute Phase of Stroke Using Lokomat Tools. Appl. Sci., 10.","DOI":"10.3390\/app10186168"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2255","DOI":"10.3390\/s120202255","article-title":"Gait Analysis Using Wearable Sensors","volume":"12","author":"Tao","year":"2012","journal-title":"Sensors"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Hssayeni, M.D., Jimenez-Shahed, J., Burack, M.A., and Ghoraani, B. (2019). Wearable Sensors for Estimation of Parkinsonian Tremor Severity during Free Body Movements. Sensors, 19.","DOI":"10.3390\/s19194215"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Marsico, M.D., and Mecca, A. (2019). A Survey on Gait Recognition via Wearable Sensors. ACM Comput. Surv., 52.","DOI":"10.1145\/3340293"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Steinert, A., Sattler, I., Otte, K., R\u00f6hling, H., Mansow-Model, S., and M\u00fcller-Werdan, U. (2020). Using New Camera-Based Technologies for Gait Analysis in Older Adults in Comparison to the Established GAITRite System. Sensors, 20.","DOI":"10.3390\/s20010125"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Rocha, A.P., Choupina, H., Fernandes, J.M., Rosas, M.J., Vaz, R., and Silva Cunha, J.P. (2015, January 25\u201329). Kinect v2 based system for Parkinson\u2019s disease assessment. Proceedings of the 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy.","DOI":"10.1109\/EMBC.2015.7318601"},{"key":"ref_12","unstructured":"Ili\u0107, T. (2015, January 6\u20139). A vision-based system for movement analysis in medical applications: The example of parkinson disease. Proceedings of the International Conference on Computer Vision Systems, Copenhagen, Denmark."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"517","DOI":"10.1016\/j.gaitpost.2010.02.014","article-title":"Solutions for representing the whole-body centre of mass in side cutting manoeuvres based on data that is typically available for lower limb kinematics","volume":"31","author":"Vanrenterghem","year":"2010","journal-title":"Gait Posture"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Serrano, M., Chen, Y., Howard, A., and Vela, P. (2016, January 16\u201320). Automated feet detection for clinical gait assessment. Proceedings of the 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA.","DOI":"10.1109\/EMBC.2016.7591157"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1553","DOI":"10.1007\/s11517-018-1795-2","article-title":"Vision-based gait impairment analysis for aided diagnosis","volume":"56","author":"Ortells","year":"2018","journal-title":"Med. Biol. Eng. Comput."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Nieto-Hidalgo, M., Ferr\u00e1ndez-Pastor, F., Valdivieso-Sarabia, R., Mora-Pascual, J., and Garc\u00eda-Chamizo, J. (2015). Vision based extraction of dynamic gait features focused on feet movement using RGB camera. Ambient Intelligence for Health, Springer.","DOI":"10.1007\/978-3-319-26508-7_16"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Leu, A., Risti\u0107-Durrant, D., and Gr\u00e4ser, A. (2011, January 19\u201321). A robust markerless vision-based human gait analysis system. Proceedings of the 2011 6th IEEE International Symposium on Applied Computational Intelligence and Informatics (SACI), Timisoara, Romania.","DOI":"10.1109\/SACI.2011.5873039"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Chaaraoui, A.A., Padilla-L\u00f3pez, J.R., and Fl\u00f3rez-Revuelta, F. (2015, January 4\u20138). Abnormal gait detection with RGB-D devices using joint motion history features. Proceedings of the 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), Ljubljana, Slovenia.","DOI":"10.1109\/FG.2015.7284881"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Nambiar, A., Correia, P.L., and Soares, L.D. (2012, January 6\u20137). Frontal Gait Recognition Combining 2D and 3D Data. Proceedings of the ACM Workshop on Multimedia and Security, Coventry, UK.","DOI":"10.1145\/2361407.2361432"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Chaparro-Rico, B.D.M., and Cafolla, D. (2020). Test-Retest, Inter-Rater and Intra-Rater Reliability for Spatiotemporal Gait Parameters Using SANE (an eaSy gAit aNalysis systEm) as Measuring Instrument. Appl. Sci., 10.","DOI":"10.3390\/app10175781"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Verlekar, T., Soares, L., and Correia, P. (2018). Automatic classification of gait impairments using a markerless 2D video-based system. Sensors, 18.","DOI":"10.3390\/s18092743"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"316","DOI":"10.1109\/TPAMI.2006.38","article-title":"Individual recognition using gait energy image","volume":"28","author":"Han","year":"2005","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Nieto-Hidalgo, M., and Garc\u00eda-Chamizo, J. (2017, January 7\u201310). Classification of pathologies using a vision based feature extraction. Proceedings of the International Conference on Ubiquitous Computing and Ambient Intelligence, Philadelphia, PA, USA.","DOI":"10.1007\/978-3-319-67585-5_28"},{"key":"ref_24","unstructured":"Zhao, G., Liu, G., Li, H., and Pietikainen, M. (2006, January 10\u201312). 3D gait recognition using multiple cameras. Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition (FGR06), Southampton, UK."},{"key":"ref_25","unstructured":"Firmino, J. (2019). Using Deep Learning for Gait Abnormality Classification. [Master\u2019s Thesis, Instituto Superior T\u00e9cnico]."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Cao, Z., Hidalgo, G., Simon, T., Wei, S.E., and Sheikh, Y. (2018). OpenPose: Realtime multi-person 2D pose estimation using Part Affinity Fields. arXiv.","DOI":"10.1109\/CVPR.2017.143"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Verlekar, T., Correia, P., and Soares, L. (2018, January 3\u20136). Using transfer learning for classification of gait pathologies. Proceedings of the 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Madrid, Spain.","DOI":"10.1109\/BIBM.2018.8621302"},{"key":"ref_28","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Li, F.F. (2009, January 20\u201325). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Loureiro, J., and Correia, P. (2020, January 16\u201320). Using a skeleton gait energy image for pathological gait classification. Proceedings of the 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020) (FG), Buenos Aires, Argentina.","DOI":"10.1109\/FG47880.2020.00064"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1016\/j.artmed.2018.12.007","article-title":"Normal and pathological gait classification LSTM model","volume":"94","author":"Khokhlova","year":"2019","journal-title":"Artif. Intell. Med."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Mutegeki, R., and Han, D.S. (2020, January 19\u201321). A CNN-LSTM approach to human activity recognition. Proceedings of the 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), Fukuoka, Japan.","DOI":"10.1109\/ICAIIC48513.2020.9065078"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Perry, J., and Schoneberger, B. (1992). Gait Analysis: Normal and Pathological Function, Slack.","DOI":"10.1097\/01241398-199211000-00023"},{"key":"ref_34","unstructured":"Albuquerque, P., Machado, J., Verlekar, T.T., Soares, L.D., and Correia, P.L. (2021). Remote Pathological Gait Classification System. arXiv."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long Short-Term Memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Li, C., Min, X., Sun, S., Lin, W., and Tang, Z. (2017). DeepGait: A learning deep convolutional representation for view-invariant gait recognition using joint Bayesian. Appl. Sci., 7.","DOI":"10.3390\/app7030210"},{"key":"ref_37","unstructured":"Latifi, S. (2018). Malicious software classification using VGG16 deep neural network\u2019s bottleneck features. Information Technology\u2014New Generations, Springer International Publishing."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Awais, M., M\u00fcller, H., Tang, T.B., and Meriaudeau, F. (2017, January 12\u201314). Classification of SD-OCT images using a deep learning approach. Proceedings of the 2017 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), Kuching, Malaysia.","DOI":"10.1109\/ICSIPA.2017.8120661"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"2673","DOI":"10.1109\/78.650093","article-title":"Bidirectional recurrent neural networks","volume":"45","author":"Schuster","year":"1997","journal-title":"Signal Process. IEEE Trans."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Singh, B., Marks, T., Jones, M., Tuzel, O., and Shao, M. (2016, January 27\u201330). A multi-stream bi-directional recurrent neural network for fine-grained action detection. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.216"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"602","DOI":"10.1016\/j.neunet.2005.06.042","article-title":"Framewise phoneme classification with bidirectional LSTM and other neural network architectures","volume":"18","author":"Graves","year":"2005","journal-title":"Neural Netw."},{"key":"ref_42","first-page":"1929","article-title":"Dropout: A Simple Way to Prevent Neural Networks from Overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_43","unstructured":"Kingma, D., and Ba, J. (2015, January 7\u20139). Adam: A method for stochastic optimization. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015\u2014Conference Track Proceedings, San Diego, CA, USA."},{"key":"ref_44","unstructured":"Dozat, T. (2016, January 2\u20134). Incorporating Nesterov Momentum into Adam. Proceedings of the 4th International Conference on Learning Representations, ICLR 2016\u2014Workshop Track Proceedings, San Juan, Puerto Rico."},{"key":"ref_45","first-page":"802","article-title":"Convolutional LSTM network: A machine learning approach for precipitation nowcasting","volume":"2015","author":"Shi","year":"2015","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1365","DOI":"10.1109\/TIFS.2020.3036242","article-title":"Long Short-Term Memory With Gate and State Level Fusion for Light Field-Based Face Recognition","volume":"16","author":"Etemad","year":"2021","journal-title":"IEEE Trans. Inf. Forensics Secur."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/18\/6202\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:00:34Z","timestamp":1760166034000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/18\/6202"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,16]]},"references-count":46,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2021,9]]}},"alternative-id":["s21186202"],"URL":"https:\/\/doi.org\/10.3390\/s21186202","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,16]]}}}