{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T16:23:35Z","timestamp":1779380615663,"version":"3.53.1"},"reference-count":48,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2023,4,7]],"date-time":"2023-04-07T00:00:00Z","timestamp":1680825600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003093","name":"Fundamental Research Grant Scheme of the Ministry of Higher Education","doi-asserted-by":"publisher","award":["FRGS\/1\/2021\/ICT02\/MMU\/02\/4"],"award-info":[{"award-number":["FRGS\/1\/2021\/ICT02\/MMU\/02\/4"]}],"id":[{"id":"10.13039\/501100003093","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003093","name":"Fundamental Research Grant Scheme of the Ministry of Higher Education","doi-asserted-by":"publisher","award":["RGP1\/357\/43"],"award-info":[{"award-number":["RGP1\/357\/43"]}],"id":[{"id":"10.13039\/501100003093","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003093","name":"Fundamental Research Grant Scheme of the Ministry of Higher Education","doi-asserted-by":"publisher","award":["MMUI\/220021"],"award-info":[{"award-number":["MMUI\/220021"]}],"id":[{"id":"10.13039\/501100003093","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003093","name":"Fundamental Research Grant Scheme of the Ministry of Higher Education","doi-asserted-by":"publisher","award":["MMU\/YUM\/C\/2019\/YPS"],"award-info":[{"award-number":["MMU\/YUM\/C\/2019\/YPS"]}],"id":[{"id":"10.13039\/501100003093","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Deanship of Scientific Research, King Khalid University, Saudi Arabia","award":["FRGS\/1\/2021\/ICT02\/MMU\/02\/4"],"award-info":[{"award-number":["FRGS\/1\/2021\/ICT02\/MMU\/02\/4"]}]},{"name":"Deanship of Scientific Research, King Khalid University, Saudi Arabia","award":["RGP1\/357\/43"],"award-info":[{"award-number":["RGP1\/357\/43"]}]},{"name":"Deanship of Scientific Research, King Khalid University, Saudi Arabia","award":["MMUI\/220021"],"award-info":[{"award-number":["MMUI\/220021"]}]},{"name":"Deanship of Scientific Research, King Khalid University, Saudi Arabia","award":["MMU\/YUM\/C\/2019\/YPS"],"award-info":[{"award-number":["MMU\/YUM\/C\/2019\/YPS"]}]},{"DOI":"10.13039\/100012024","name":"Multimedia University Internal Research Grant","doi-asserted-by":"publisher","award":["FRGS\/1\/2021\/ICT02\/MMU\/02\/4"],"award-info":[{"award-number":["FRGS\/1\/2021\/ICT02\/MMU\/02\/4"]}],"id":[{"id":"10.13039\/100012024","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100012024","name":"Multimedia University Internal Research Grant","doi-asserted-by":"publisher","award":["RGP1\/357\/43"],"award-info":[{"award-number":["RGP1\/357\/43"]}],"id":[{"id":"10.13039\/100012024","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100012024","name":"Multimedia University Internal Research Grant","doi-asserted-by":"publisher","award":["MMUI\/220021"],"award-info":[{"award-number":["MMUI\/220021"]}],"id":[{"id":"10.13039\/100012024","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100012024","name":"Multimedia University Internal Research Grant","doi-asserted-by":"publisher","award":["MMU\/YUM\/C\/2019\/YPS"],"award-info":[{"award-number":["MMU\/YUM\/C\/2019\/YPS"]}],"id":[{"id":"10.13039\/100012024","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Yayasan Universiti Multimedia","award":["FRGS\/1\/2021\/ICT02\/MMU\/02\/4"],"award-info":[{"award-number":["FRGS\/1\/2021\/ICT02\/MMU\/02\/4"]}]},{"name":"Yayasan Universiti Multimedia","award":["RGP1\/357\/43"],"award-info":[{"award-number":["RGP1\/357\/43"]}]},{"name":"Yayasan Universiti Multimedia","award":["MMUI\/220021"],"award-info":[{"award-number":["MMUI\/220021"]}]},{"name":"Yayasan Universiti Multimedia","award":["MMU\/YUM\/C\/2019\/YPS"],"award-info":[{"award-number":["MMU\/YUM\/C\/2019\/YPS"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Gait recognition, the task of identifying an individual based on their unique walking style, can be difficult because walking styles can be influenced by external factors such as clothing, viewing angle, and carrying conditions. To address these challenges, this paper proposes a multi-model gait recognition system that integrates Convolutional Neural Networks (CNNs) and Vision Transformer. The first step in the process is to obtain a gait energy image, which is achieved by applying an averaging technique to a gait cycle. The gait energy image is then fed into three different models, DenseNet-201, VGG-16, and a Vision Transformer. These models are pre-trained and fine-tuned to encode the salient gait features that are specific to an individual\u2019s walking style. Each model provides prediction scores for the classes based on the encoded features, and these scores are then summed and averaged to produce the final class label. The performance of this multi-model gait recognition system was evaluated on three datasets, CASIA-B, OU-ISIR dataset D, and OU-ISIR Large Population dataset. The experimental results showed substantial improvement compared to existing methods on all three datasets. The integration of CNNs and ViT allows the system to learn both the pre-defined and distinct features, providing a robust solution for gait recognition even under the influence of covariates.<\/jats:p>","DOI":"10.3390\/s23083809","type":"journal-article","created":{"date-parts":[[2023,4,10]],"date-time":"2023-04-10T03:24:18Z","timestamp":1681097058000},"page":"3809","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":51,"title":["Gait-CNN-ViT: Multi-Model Gait Recognition with Convolutional Neural Networks and Vision Transformer"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7873-0033","authenticated-orcid":false,"given":"Jashila Nair","family":"Mogan","sequence":"first","affiliation":[{"name":"Faculty of Information Science and Technology, Multimedia University, Melaka 75450, Malaysia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3679-8977","authenticated-orcid":false,"given":"Chin Poo","family":"Lee","sequence":"additional","affiliation":[{"name":"Faculty of Information Science and Technology, Multimedia University, Melaka 75450, Malaysia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1929-7978","authenticated-orcid":false,"given":"Kian Ming","family":"Lim","sequence":"additional","affiliation":[{"name":"Faculty of Information Science and Technology, Multimedia University, Melaka 75450, Malaysia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5908-4013","authenticated-orcid":false,"given":"Mohammed","family":"Ali","sequence":"additional","affiliation":[{"name":"Department of Computer Science, King Khalid University, Abha 61421, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1052-2657","authenticated-orcid":false,"given":"Ali","family":"Alqahtani","sequence":"additional","affiliation":[{"name":"Department of Computer Science, King Khalid University, Abha 61421, Saudi Arabia"},{"name":"Center for Artificial Intelligence (CAI), King Khalid University, Abha 61421, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,4,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_2","first-page":"1106","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21\u201326). Densely connected convolutional networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_4","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Ahmed, M., Al-Jawad, N., and Sabir, A.T. (2014, January 16\u201317). Gait recognition based on Kinect sensor. Proceedings of the Real-Time Image and Video Processing 2014, SPIE, Brussels, Belgium.","DOI":"10.1117\/12.2052588"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Sattrupai, T., and Kusakunniran, W. (2018, January 28\u201331). Deep trajectory based gait recognition for human re-identification. Proceedings of the TENCON 2018\u20142018 IEEE Region 10 Conference, Jeju Island, Republic of Korea.","DOI":"10.1109\/TENCON.2018.8650523"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2577","DOI":"10.1109\/TIFS.2019.2901823","article-title":"Skeleton-based gait recognition via robust frame-level matching","volume":"14","author":"Choi","year":"2019","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"497","DOI":"10.1007\/s10044-020-00935-z","article-title":"Simple and efficient pose-based gait recognition method for challenging environments","volume":"24","author":"Melo","year":"2021","journal-title":"Pattern Anal. Appl."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"186","DOI":"10.1016\/j.patcog.2017.02.014","article-title":"Fusion of spatial-temporal and kinematic features for gait recognition with deterministic learning","volume":"67","author":"Deng","year":"2017","journal-title":"Pattern Recognit."},{"key":"ref_10","unstructured":"Sah, S., and Panday, S.P. (2020, January 5\u20137). Model Based Gait Recognition Using Weighted KNN. Proceedings of the 8th IOE Graduate Conference, Kathmandu, Nepal."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"67","DOI":"10.4018\/JOEUC.2020040104","article-title":"A machine learning method with threshold based parallel feature fusion and feature selection for automated gait recognition","volume":"32","author":"Sharif","year":"2020","journal-title":"J. Organ. End User Comput. (JOEUC)"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1489","DOI":"10.1016\/j.jvcir.2014.05.006","article-title":"Gait probability image: An information-theoretic model of gait representation","volume":"25","author":"Lee","year":"2014","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"822","DOI":"10.1016\/j.jvcir.2014.01.012","article-title":"Time-sliced averaged motion history image for gait recognition","volume":"25","author":"Lee","year":"2014","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Mogan, J.N., Lee, C.P., and Tan, A.W. (2017, January 3\u20135). Gait recognition using temporal gradient patterns. Proceedings of the 2017 5th International Conference on Information and Communication Technology (ICoIC7), Malacca, Malaysia.","DOI":"10.1109\/ICoICT.2017.8074680"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Mogan, J.N., Lee, C.P., Lim, K.M., and Tan, A.W. (2017, January 27\u201329). Gait recognition using binarized statistical image features and histograms of oriented gradients. Proceedings of the 2017 International Conference on Robotics, Automation and Sciences (ICORAS), Melaka, Malaysia.","DOI":"10.1109\/ICORAS.2017.8308067"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"3601","DOI":"10.1007\/s13042-019-00947-0","article-title":"Multi-level features fusion and selection for human gait recognition: An optimized framework of Bayesian model and binomial distribution","volume":"10","author":"Arshad","year":"2019","journal-title":"Int. J. Mach. Learn. Cybern."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"252","DOI":"10.3844\/ajassp.2017.252.266","article-title":"Review on vision-based gait recognition: Representations, classification schemes and datasets","volume":"14","author":"Lee","year":"2017","journal-title":"Am. J. Appl. Sci."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"663","DOI":"10.1016\/j.patrec.2013.01.013","article-title":"Gait recognition via optimally interpolated deformable contours","volume":"34","author":"Lee","year":"2013","journal-title":"Pattern Recognit. Lett."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1016\/j.jvcir.2015.09.006","article-title":"Gait recognition with transient binary patterns","volume":"33","author":"Lee","year":"2015","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1007\/s11760-018-1365-y","article-title":"Spatiotemporal features of human motion for gait recognition","volume":"13","author":"Khan","year":"2019","journal-title":"Signal Image Video Process."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"012051","DOI":"10.1088\/1742-6596\/1502\/1\/012051","article-title":"Gait recognition using histograms of temporal gradients","volume":"1502","author":"Mogan","year":"2020","journal-title":"J. Phys. Conf. Ser."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"424","DOI":"10.1016\/j.jvcir.2018.06.019","article-title":"Feedback weight convolutional neural network for gait recognition","volume":"55","author":"Wu","year":"2018","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"14275","DOI":"10.1007\/s00521-019-04524-y","article-title":"Gait recognition using multichannel convolution neural networks","volume":"32","author":"Wang","year":"2020","journal-title":"Neural Comput. Appl."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"115057","DOI":"10.1016\/j.eswa.2021.115057","article-title":"Multi-view gait recognition system using spatio-temporal features and deep learning","volume":"179","author":"Gul","year":"2021","journal-title":"Expert Syst. Appl."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"108519","DOI":"10.1016\/j.patcog.2021.108519","article-title":"A unified perspective of classification-based loss and distance-based loss for cross-view gait recognition","volume":"125","author":"Han","year":"2022","journal-title":"Pattern Recognit."},{"key":"ref_26","first-page":"3467","article-title":"GaitSet: Cross-view gait recognition through utilizing gait as a deep set","volume":"44","author":"Chao","year":"2021","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Li, C., Min, X., Sun, S., Lin, W., and Tang, Z. (2017). DeepGait: A learning deep convolutional representation for view-invariant gait recognition using joint bayesian. Appl. Sci., 7.","DOI":"10.3390\/app7030210"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"e12541","DOI":"10.1111\/exsy.12541","article-title":"A multilevel paradigm for deep convolutional neural network features selection with an application to human gait recognition","volume":"39","author":"Arshad","year":"2020","journal-title":"Expert Syst."},{"key":"ref_29","first-page":"343","article-title":"Human Gait Recognition: A Deep Learning and Best Feature Selection Framework","volume":"70","author":"Mehmood","year":"2022","journal-title":"Comput. Mater. Cont"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1007\/s11760-018-1326-5","article-title":"On the selection of spatiotemporal filtering with classifier ensemble method for effective gait recognition","volume":"13","author":"Ghaeminia","year":"2019","journal-title":"Signal Image Video Process."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1565","DOI":"10.1007\/s11042-020-09777-7","article-title":"Gait classification through CNN-based ensemble learning","volume":"80","author":"Wang","year":"2021","journal-title":"Multimed. Tools Appl."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"3102","DOI":"10.1109\/TIFS.2019.2912577","article-title":"Joint intensity transformer network for gait recognition robust against clothing and carrying status","volume":"14","author":"Li","year":"2019","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"260","DOI":"10.1109\/TCSVT.2020.2975671","article-title":"Cross-view gait recognition using pairwise spatial transformer networks","volume":"31","author":"Xu","year":"2020","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"6065","DOI":"10.1007\/s11042-020-09935-x","article-title":"Non-local gait feature extraction and human identification","volume":"80","author":"Wang","year":"2021","journal-title":"Multimed. Tools Appl."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Pin\u010di\u0107, D., Su\u0161anj, D., and Lenac, K. (2022). Gait Recognition with Self-Supervised Learning of Gait Features Based on Vision Transformers. Sensors, 22.","DOI":"10.3390\/s22197140"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Mogan, J.N., Lee, C.P., Lim, K.M., and Muthu, K.S. (2022). Gait-ViT: Gait Recognition with Vision Transformer. Sensors, 22.","DOI":"10.3390\/s22197362"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"316","DOI":"10.1109\/TPAMI.2006.38","article-title":"Individual recognition using gait energy image","volume":"28","author":"Han","year":"2005","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_38","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv."},{"key":"ref_39","unstructured":"Yu, S., Tan, D., and Tan, T. (2006, January 20\u201324). A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition. Proceedings of the 18th International Conference on Pattern Recognition (ICPR\u201906), Hong Kong, China."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"53","DOI":"10.2197\/ipsjtcva.4.53","article-title":"The OU-ISIR gait database comprising the treadmill dataset","volume":"4","author":"Makihara","year":"2012","journal-title":"IPSJ Trans. Comput. Vis. Appl."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1511","DOI":"10.1109\/TIFS.2012.2204253","article-title":"The OU-ISIR gait database comprising the large population dataset and performance evaluation of gait recognition","volume":"7","author":"Iwama","year":"2012","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Shiraga, K., Makihara, Y., Muramatsu, D., Echigo, T., and Yagi, Y. (2016, January 13\u201316). Geinet: View-invariant gait recognition using a convolutional neural network. Proceedings of the 2016 International Conference on Biometrics (ICB), Halmstad, Sweden.","DOI":"10.1109\/ICB.2016.7550060"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1016\/j.cviu.2017.10.004","article-title":"Improved gait recognition based on specialized deep convolutional neural network","volume":"164","author":"Alotaibi","year":"2017","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Min, P.P., Sayeed, S., and Ong, T.S. (2019, January 24\u201326). Gait recognition using deep convolutional features. Proceedings of the 2019 7th International Conference on Information and Communication Technology (ICoICT), Kuala Lumpur, Malaysia.","DOI":"10.1109\/ICoICT.2019.8835194"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Aung, H.M.L., and Pluempitiwiriyawej, C. (2020, January 15\u201317). Gait Biometric-based Human Recognition System Using Deep Convolutional Neural Network in Surveillance System. Proceedings of the 2020 Asia Conference on Computers and Communications (ACCC), Shanghai, China.","DOI":"10.1109\/ACCC51160.2020.9347899"},{"key":"ref_46","first-page":"472","article-title":"Deep Features Based Multiview Gait Recognition","volume":"12","author":"Balamurugan","year":"2021","journal-title":"Turk. J. Comput. Math. Educ. (TURCOMAT)"},{"key":"ref_47","first-page":"2113","article-title":"Human Gait Recognition Using Deep Learning and Improved Ant Colony Optimization","volume":"70","author":"Khan","year":"2022","journal-title":"CMC-Comput. Mater. Contin."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"7275","DOI":"10.1007\/s00521-019-04256-z","article-title":"Cross-view gait recognition through ensemble learning","volume":"32","author":"Wang","year":"2020","journal-title":"Neural Comput. Appl."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/8\/3809\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:12:07Z","timestamp":1760123527000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/8\/3809"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,7]]},"references-count":48,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2023,4]]}},"alternative-id":["s23083809"],"URL":"https:\/\/doi.org\/10.3390\/s23083809","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,7]]}}}