{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:56:49Z","timestamp":1760147809077,"version":"build-2065373602"},"reference-count":35,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2023,3,2]],"date-time":"2023-03-02T00:00:00Z","timestamp":1677715200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"\u201cHuman Resources Program in Energy Technology\u201d of the Korea Institute of Energy Technology Evaluation and Planning (KETEP)","award":["20204010600090","PSAU\/2023\/R\/1444"],"award-info":[{"award-number":["20204010600090","PSAU\/2023\/R\/1444"]}]},{"name":"Ministry of Trade, Industry &amp; Energy, Republic of Korea","award":["20204010600090","PSAU\/2023\/R\/1444"],"award-info":[{"award-number":["20204010600090","PSAU\/2023\/R\/1444"]}]},{"name":"Prince Satam bin Abdulaziz University","award":["20204010600090","PSAU\/2023\/R\/1444"],"award-info":[{"award-number":["20204010600090","PSAU\/2023\/R\/1444"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The performance of human gait recognition (HGR) is affected by the partial obstruction of the human body caused by the limited field of view in video surveillance. The traditional method required the bounding box to recognize human gait in the video sequences accurately; however, it is a challenging and time-consuming approach. Due to important applications, such as biometrics and video surveillance, HGR has improved performance over the last half-decade. Based on the literature, the challenging covariant factors that degrade gait recognition performance include walking while wearing a coat or carrying a bag. This paper proposed a new two-stream deep learning framework for human gait recognition. The first step proposed a contrast enhancement technique based on the local and global filters information fusion. The high-boost operation is finally applied to highlight the human region in a video frame. Data augmentation is performed in the second step to increase the dimension of the preprocessed dataset (CASIA-B). In the third step, two pre-trained deep learning models\u2014MobilenetV2 and ShuffleNet\u2014are fine-tuned and trained on the augmented dataset using deep transfer learning. Features are extracted from the global average pooling layer instead of the fully connected layer. In the fourth step, extracted features of both streams are fused using a serial-based approach and further refined in the fifth step by using an improved equilibrium state optimization-controlled Newton\u2013Raphson (ESOcNR) selection method. The selected features are finally classified using machine learning algorithms for the final classification accuracy. The experimental process was conducted on 8 angles of the CASIA-B dataset and obtained an accuracy of 97.3, 98.6, 97.7, 96.5, 92.9, 93.7, 94.7, and 91.2%, respectively. Comparisons were conducted with state-of-the-art (SOTA) techniques, and showed improved accuracy and reduced computational time.<\/jats:p>","DOI":"10.3390\/s23052754","type":"journal-article","created":{"date-parts":[[2023,3,3]],"date-time":"2023-03-03T02:03:08Z","timestamp":1677808988000},"page":"2754","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["A Fusion-Assisted Multi-Stream Deep Learning and ESO-Controlled Newton\u2013Raphson-Based Feature Selection Approach for Human Gait Recognition"],"prefix":"10.3390","volume":"23","author":[{"given":"Faiza","family":"Jahangir","sequence":"first","affiliation":[{"name":"Department of Computer Science, HITEC University, Taxila 47080, Pakistan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6347-4890","authenticated-orcid":false,"given":"Muhammad Attique","family":"Khan","sequence":"additional","affiliation":[{"name":"Department of Computer Science, HITEC University, Taxila 47080, Pakistan"},{"name":"Department of Informatics, University of Leicester, Leicester LE1 7RH, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Majed","family":"Alhaisoni","sequence":"additional","affiliation":[{"name":"College of Computer Science and Engineering, University of Ha\u2019il, Ha\u2019il 81451, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Abdullah","family":"Alqahtani","sequence":"additional","affiliation":[{"name":"Department of Software Engineering, College of Computer Engineering and Sciences, Prince Sattam bin Abdulaziz University, Al-kharj 16242, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6584-7400","authenticated-orcid":false,"given":"Shtwai","family":"Alsubai","sequence":"additional","affiliation":[{"name":"Department of Computer Science, College of Computer Engineering and Sciences, Prince Sattam bin Abdulaziz University, Al-kharj 16242, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mohemmed","family":"Sha","sequence":"additional","affiliation":[{"name":"Department of Software Engineering, College of Computer Engineering and Sciences, Prince Sattam bin Abdulaziz University, Al-kharj 16242, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Abdullah","family":"Al Hejaili","sequence":"additional","affiliation":[{"name":"Faculty of Computers & Information Technology, Computer Science Department, University of Tabuk, Tabuk 71491, Saudi Arabia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jae-hyuk","family":"Cha","sequence":"additional","affiliation":[{"name":"Department of computer Science, Hanyang University, Seoul 04763, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,3,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Jain, A.K., Ross, A., and Prabhakar, S. (2004). An Introduction to Biometric Recognition, IEEE.","DOI":"10.1109\/TCSVT.2003.818349"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1109\/TIFS.2006.873653","article-title":"Biometrics: A tool for information security","volume":"1","author":"Jain","year":"2006","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Liu, L., Wang, H., Li, H., Liu, J., Qiu, S., Zhao, H., and Guo, X. (2021). Ambulatory human gait phase detection using wearable inertial sensors and hidden Markov model. Sensors, 21.","DOI":"10.3390\/s21041347"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2302","DOI":"10.1109\/JBHI.2019.2938111","article-title":"From emotions to mood disorders: A survey on gait analysis methodology","volume":"23","author":"Deligianni","year":"2019","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"514","DOI":"10.1016\/j.neucom.2022.06.048","article-title":"PoseMapGait: A model-based gait recognition method with pose estimation maps and graph convolutional networks","volume":"501","author":"Liao","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_6","unstructured":"Gafurov, D. (2007, January 20\u201324). A survey of biometric gait recognition: Approaches, security and challenges. Proceedings of the Annual Norwegian Computer Science Conference, Bergen, Norway."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Khan, M.A., Kadry, S., Parwekar, P., Dama\u0161evi\u010dius, R., Mehmood, A., Khan, J.A., and Naqvi, S.R. (2021). Human gait analysis for osteoarthritis prediction: A framework of deep learning and kernel extreme learning machine. Complex Intell. Syst., 1\u201319.","DOI":"10.1007\/s40747-020-00244-2"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"14213","DOI":"10.1109\/JSEN.2021.3066473","article-title":"Fusion of multi-sensor-based biomechanical gait analysis using vision and wearable sensor","volume":"21","author":"Bijalwan","year":"2021","journal-title":"IEEE Sens. J."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Bayat, N., Rastegari, E., and Li, Q. (2022). Human Gait Recognition Using Bag of Words Feature Representation Method. arXiv.","DOI":"10.54941\/ahfe1001481"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Derlatka, M., and Borowska, M. (2023). Ensemble of heterogeneous base classifiers for human gait recognition. Sensors, 23.","DOI":"10.3390\/s23010508"},{"key":"ref_11","unstructured":"Turk, M.A., and Pentland, A.P. (1991, January 3\u20136). Face recognition using eigenfaces. Proceedings of the 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Maui, HI, USA."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1049\/iet-cvi.2009.0009","article-title":"Gait recognition using active shape model and motion prediction","volume":"4","author":"Kim","year":"2010","journal-title":"IET Comput. Vision"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Wang, L., Li, Y., Xiong, F., and Zhang, W. (2021). Gait Recognition Using Optical Motion Capture: A Decision Fusion Based Method. Sensors, 21.","DOI":"10.3390\/s21103496"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1016\/j.neucom.2021.04.081","article-title":"A novel view synthesis approach based on view space covering for gait recognition","volume":"453","author":"Liao","year":"2021","journal-title":"Neurocomputing"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Slemen\u0161ek, J., Fister, I., Ger\u0161ak, J., Bratina, B., van Midden, V.M., Pirto\u0161ek, Z., and \u0160afari\u010d, R. (2023). Human Gait Activity Recognition Machine Learning Methods. Sensors, 23.","DOI":"10.3390\/s23020745"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Pin\u010di\u0107, D., Su\u0161anj, D., and Lenac, K. (2022). Gait Recognition with Self-Supervised Learning of Gait Features Based on Vision Transformers. Sensors, 22.","DOI":"10.3390\/s22197140"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3230633","article-title":"A survey on gait recognition","volume":"51","author":"Wan","year":"2018","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1309","DOI":"10.1109\/TIFS.2023.3236181","article-title":"Occlusion-aware Human Mesh Model-based Gait Recognition","volume":"18","author":"Xu","year":"2023","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Zhu, H., Zheng, Z., and Nevatia, R. (2023, January 2\u20137). Gait Recognition Using 3-D Human Body Shape Inference. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.","DOI":"10.1109\/WACV56688.2023.00097"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Shi, L.-F., Liu, Z.-Y., Zhou, K.-J., Shi, Y., and Jing, X. (2023). Novel Deep Learning Network for Gait Recognition Using Multimodal Inertial Sensors. Sensors, 23.","DOI":"10.3390\/s23020849"},{"key":"ref_21","first-page":"931","article-title":"Implementation of machine learning algorithms for gait recognition","volume":"23","author":"Kececi","year":"2020","journal-title":"Eng. Sci. Technol. Int. J."},{"key":"ref_22","unstructured":"Dou, H., Zhang, W., Zhang, P., Zhao, Y., Li, S., Qin, Z., Wu, F., Dong, L., and Li, X. (2021). VersatileGait: A large-scale synthetic gait dataset with fine-Grained Attributes and complicated scenarios. arXiv."},{"key":"ref_23","unstructured":"Zhu, Z., Guo, X., Yang, T., Huang, J., Deng, J., Huang, G., Du, D., Lu, J., and Zhou, J. (2021, January 11\u201317). Gait recognition in the wild: A benchmark. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Huang, C., Zhang, F., Xu, Z., and Wei, J. (2022). The Diverse Gait Dataset: Gait segmentation using inertial sensors for pedestrian localization with different genders, heights and walking speeds. Sensors, 22.","DOI":"10.3390\/s22041678"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"42577","DOI":"10.1109\/ACCESS.2022.3168019","article-title":"Gait recognition with wearable sensors using modified residual block-based lightweight cnn","volume":"10","author":"Hasan","year":"2022","journal-title":"IEEE Access"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"421","DOI":"10.1109\/TBIOM.2020.3008862","article-title":"Performance evaluation of model-based gait on multi-view very large population database with pose sequences","volume":"2","author":"An","year":"2020","journal-title":"IEEE Trans. Biom. Behav. Identity Sci."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Tian, Y., Wei, L., Lu, S., and Huang, T. (2019). Free-view gait recognition. PLoS ONE, 14.","DOI":"10.1371\/journal.pone.0214389"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"105190","DOI":"10.1016\/j.knosys.2019.105190","article-title":"Equilibrium optimizer: A novel optimization algorithm","volume":"191","author":"Faramarzi","year":"2020","journal-title":"Knowl. Based Syst."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"e06673","DOI":"10.1016\/j.heliyon.2021.e06673","article-title":"A combination of Newton-Raphson method and heuristics algorithms for parameter estimation in photovoltaic modules","volume":"7","author":"Gnetchejo","year":"2021","journal-title":"Heliyon"},{"key":"ref_30","first-page":"2113","article-title":"Human gait recognition using deep learning and improved ant colony optimization","volume":"70","author":"Khan","year":"2022","journal-title":"Comput. Mater. Contin."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"100556","DOI":"10.1016\/j.rineng.2022.100556","article-title":"Human gait recognition subject to different covariate factors in a multi-view environment","volume":"15","author":"Asif","year":"2022","journal-title":"Results Eng."},{"key":"ref_32","unstructured":"Wang, L., Zhang, X., Han, R., Yang, J., Li, X., Feng, W., and Wang, S. (2022). A Benchmark of Video-Based Clothes-Changing Person Re-Identification. arXiv."},{"key":"ref_33","unstructured":"Rao, P.S., Sahu, G., Parida, P., and Patnaik, S. (2022). Smart and Sustainable Technologies: Rural and Tribal Development Using IoT and Cloud Computing: Proceedings of ICSST 2021, Springer."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Mehmood, A., Khan, M.A., Sharif, M., Khan, S.A., Shaheen, M., Saba, T., Riaz, N., and Ashraf, I. (2020). Prosperous human gait recognition: An end-to-end system based on pre-trained CNN features selection. Multimed. Tools Appl., 1\u201321.","DOI":"10.1007\/s11042-020-08928-0"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"2873","DOI":"10.1007\/s11042-019-08400-8","article-title":"Clothing invariant human gait recognition using modified local optimal oriented pattern binary descriptor","volume":"79","author":"Anusha","year":"2020","journal-title":"Multimed. Tools Appl."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/5\/2754\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:46:24Z","timestamp":1760121984000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/5\/2754"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,2]]},"references-count":35,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["s23052754"],"URL":"https:\/\/doi.org\/10.3390\/s23052754","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2023,3,2]]}}}