{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T09:32:38Z","timestamp":1778578358920,"version":"3.51.4"},"reference-count":41,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2017,11,27]],"date-time":"2017-11-27T00:00:00Z","timestamp":1511740800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The wide spread usage of wearable sensors such as in smart watches has provided continuous access to valuable user generated data such as human motion that could be used to identify an individual based on his\/her motion patterns such as, gait. Several methods have been suggested to extract various heuristic and high-level features from gait motion data to identify discriminative gait signatures and distinguish the target individual from others. However, the manual and hand crafted feature extraction is error prone and subjective. Furthermore, the motion data collected from inertial sensors have complex structure and the detachment between manual feature extraction module and the predictive learning models might limit the generalization capabilities. In this paper, we propose a novel approach for human gait identification using time-frequency (TF) expansion of human gait cycles in order to capture joint 2 dimensional (2D) spectral and temporal patterns of gait cycles. Then, we design a deep convolutional neural network (DCNN) learning to extract discriminative features from the 2D expanded gait cycles and jointly optimize the identification model and the spectro-temporal features in a discriminative fashion. We collect raw motion data from five inertial sensors placed at the chest, lower-back, right hand wrist, right knee, and right ankle of each human subject synchronously in order to investigate the impact of sensor location on the gait identification performance. We then present two methods for early (input level) and late (decision score level) multi-sensor fusion to improve the gait identification generalization performance. We specifically propose the minimum error score fusion (MESF) method that discriminatively learns the linear fusion weights of individual DCNN scores at the decision level by minimizing the error rate on the training data in an iterative manner. 10 subjects participated in this study and hence, the problem is a 10-class identification task. Based on our experimental results, 91% subject identification accuracy was achieved using the best individual IMU and 2DTF-DCNN. We then investigated our proposed early and late sensor fusion approaches, which improved the gait identification accuracy of the system to 93.36% and 97.06%, respectively.<\/jats:p>","DOI":"10.3390\/s17122735","type":"journal-article","created":{"date-parts":[[2017,11,27]],"date-time":"2017-11-27T11:07:08Z","timestamp":1511780828000},"page":"2735","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":169,"title":["IMU-Based Gait Recognition Using Convolutional Neural Networks and Multi-Sensor Fusion"],"prefix":"10.3390","volume":"17","author":[{"given":"Omid","family":"Dehzangi","sequence":"first","affiliation":[{"name":"Computer and Information Science Department, University of Michigan-Dearborn, Dearborn, MI 48128, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5534-5776","authenticated-orcid":false,"given":"Mojtaba","family":"Taherisadr","sequence":"additional","affiliation":[{"name":"Computer and Information Science Department, University of Michigan-Dearborn, Dearborn, MI 48128, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5665-2401","authenticated-orcid":false,"given":"Raghvendar","family":"ChangalVala","sequence":"additional","affiliation":[{"name":"Computer and Information Science Department, University of Michigan-Dearborn, Dearborn, MI 48128, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2017,11,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"335","DOI":"10.2106\/00004623-196446020-00009","article-title":"Walking patterns of normal men","volume":"46","author":"Murray","year":"1964","journal-title":"JBJS"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"142","DOI":"10.1007\/BF00688098","article-title":"Neuropathology of Rett syndrome","volume":"76","author":"Jellinger","year":"1988","journal-title":"Acta Neuropathol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1236","DOI":"10.1002\/art.1780380910","article-title":"Degenerative lumbar spinal stenosis Diagnostic value of the history and physical examination","volume":"38","author":"Katz","year":"1995","journal-title":"Arthritis Rheumatol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"268","DOI":"10.1212\/WNL.43.2.268","article-title":"Human walking and higher-level gait disorders, particularly in the elderly","volume":"43","author":"Nutt","year":"1993","journal-title":"Neurology"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1109\/TIM.2007.908635","article-title":"Analysis and modeling of inertial sensors using Allan variance","volume":"57","author":"Haiying","year":"2008","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1163","DOI":"10.1109\/TIP.2004.832865","article-title":"Identification of humans using gait","volume":"13","author":"Kale","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"22089","DOI":"10.3390\/s150922089","article-title":"Inertial sensor-based gait recognition: A review","volume":"15","author":"Sprager","year":"2015","journal-title":"Sensors"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Gafurov, D., Einar, S., and Patrick, B. (2007, January 7\u20138). Gait authentication and identification using wearable accelerometer sensor. Proceedings of the IEEE Workshop on Automatic Identification Advanced Technologies, Alghero, Italy.","DOI":"10.1109\/AUTOID.2007.380623"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1109\/MPRV.2010.7","article-title":"Human activity recognition and pattern discovery","volume":"9","author":"Kim","year":"2010","journal-title":"IEEE Pervasive Comput."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Mortazavi, B., Alsharufa, N., Lee, S.I., Lan, M., Sarrafzadeh, M., Chronley, M., and Roberts, C.K. (2013, January 6\u20139). MET calculations from on-body accelerometers for exergaming movements. Proceedings of the IEEE International Conference on Body Sensor Networks (BSN), Cambridge, MA, USA.","DOI":"10.1109\/BSN.2013.6575520"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Vikas, V., and Carl, D.C. (2013, January 4\u20137). Measurement of robot link joint parameters using multiple accelerometers and gyroscopes. Proceedings of the ASME 2013 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, Portland, OR, USA.","DOI":"10.1115\/DETC2013-12741"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"598","DOI":"10.1093\/arclin\/acu038.247","article-title":"C-66 Prompting Technologies: Is Prompting during Activity Transition More Effective than Time-Based Prompting?","volume":"29","author":"Robertson","year":"2014","journal-title":"Arch. Clin. Neuropsychol."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ahmadi, A., Edmond, M., Francois, D., Marc, G., Noel, E.O., Chris, R., and Kieran, M. (2014, January 16\u201319). Automatic activity classification and movement assessment during a sports training session using wearable inertial sensors. Proceedings of the International Conference on IEEE Wearable and Implantable Body Sensor Networks (BSN), Zurich, Switzerland.","DOI":"10.1109\/BSN.2014.29"},{"key":"ref_14","unstructured":"Le, M.J., and Ingo, S. (2015, January 9\u201311). The smartphone as a gait recognition device impact of selected parameters on gait recognition. Proceedings of the 2015 International Conference on Information Systems Security and Privacy (ICISSP), Angers, France."},{"key":"ref_15","unstructured":"Wang, Y., and Kobsa, A. (2008). Handbook of Research on Social and Organizational Liabilities in Information Security, Chapter Privacy Enhancing Technology, Idea Group Inc. Global."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"4405","DOI":"10.1007\/s11042-015-3177-1","article-title":"A survey of depth and inertial sensor fusion for human action recognition","volume":"76","author":"Chen","year":"2017","journal-title":"Multimed. Tools Appl."},{"key":"ref_17","unstructured":"Roberts, M.L., and Debra, Z. (2012). Internet Marketing: Integrating Online and Offline Strategies, Cengage Learning."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3869","DOI":"10.1007\/s00296-011-2283-2","article-title":"Objective assessment of abnormal gait in patients with rheumatoid arthritis using a smartphone","volume":"32","author":"Yamada","year":"2012","journal-title":"Rheumatol. Int."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Sposaro, F., and Gary, T. (2009, January 3\u20136). iFall: An Android application for fall monitoring and response. Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Minneapolis, MN, USA.","DOI":"10.1109\/IEMBS.2009.5334912"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Yurtman, A., and Billur, B. (2017). Activity Recognition Invariant to Sensor Orientation with Wearable Motion Sensors. Sensors, 17.","DOI":"10.3390\/s17081838"},{"key":"ref_21","unstructured":"Mantyjarvi, J., Lindholm, M., Vildjiounaite, E., Makela, S.-M., and Ailisto, H.A. (2005, January 23\u201323). Identifying users of portable devices from gait pattern with accelerometers. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP\u201905), Philadelphia, PA, USA."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Subramanian, R., Sudeep, S., Miguel, L., Kristina, C., Christopher, E., Omar, J., Jiejie, Z., and Hui, C. (2015, January 23\u201325). Orientation invariant gait matching algorithm based on the Kabsch alignment. Proceedings of the IEEE International Conference on Identity, Security and Behavior Analysis (ISBA 2015), Hong Kong, China.","DOI":"10.1109\/ISBA.2015.7126347"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Gafurov, D., Einar, S., and Patrick, B. (2010, January 20\u201323). Improved gait recognition performance using cycle matching. Proceedings of the 24th International Conference on Advanced Information Networking and Applications Workshops, Perth, Australia.","DOI":"10.1109\/WAINA.2010.145"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Rong, L., Duan, Z., Zhou, J., and Liu, M. (2007, January 6\u20138). Identification of individual walking patterns using gait acceleration. Proceedings of the 2007 1st International Conference on Bioinformatics and Biomedical Engineering, Wuhan, China.","DOI":"10.1109\/ICBBE.2007.142"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Kwapisz, J.R., Gary, M.W., and Samuel, A.M. (2010, January 27\u201329). Cell phone-based biometric identification. Proceedings of the 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS), Washington, DC, USA.","DOI":"10.1109\/BTAS.2010.5634532"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Thang, H.M., Vo Quang, V., Nguyen, D.T., and Deokjai, C. (2012, January 26\u201329). Gait identification using accelerometer on mobile phone. Proceedings of the 2012 International Conference on Control, Automation and Information Sciences (ICCAIS), Ho Chi Minh City, Vietnam.","DOI":"10.1109\/ICCAIS.2012.6466615"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1486","DOI":"10.1109\/TIFS.2015.2415753","article-title":"An efficient HOS-based gait authentication of accelerometer data","volume":"10","author":"Sprager","year":"2015","journal-title":"IEEE Trans. Inform. Forensics Secur."},{"key":"ref_28","unstructured":"Hrushikesh, M., Qianli, L., and Tomaso, P. (2017, January 4\u20139). When and Why Are Deep Networks Better Than Shallow Ones?. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, San Francisco, CA, USA."},{"key":"ref_29","unstructured":"(2017, November 27). SHIMMER Getting Started Guide. Available online: http:\/\/www.eecs.harvard.edu\/~konrad\/projects\/shimmer\/SHIMMER-GettingStartedGuide.html."},{"key":"ref_30","unstructured":"Baharev, A. (2017, November 27). Gyroscope Calibration. Available online: http:\/\/reliablecomputing.eu\/baharev-gyro.pdf."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"540","DOI":"10.1109\/5.135378","article-title":"Estimating and interpreting the instantaneous frequency of a signal. II. Algorithms and applications","volume":"80","author":"Boashash","year":"1992","journal-title":"Proc. IEEE"},{"key":"ref_32","unstructured":"Pedersen, F. (1997). Joint Time Frequency Analysis in Digital Signal Processing, Aalborg Universitetsforlag."},{"key":"ref_33","unstructured":"Karpathy, A. (2016). Cs231n: Convolutional Neural Networks for Visual Recognition. Neural Netw., Available online: http:\/\/cs231n.github.io\/."},{"key":"ref_34","unstructured":"Zhong, Y., and Yunbin, D. (Octomber, January 29). Sensor orientation invariant mobile gait biometrics. Proceedings of the IEEE International Joint Conference on Biometrics, Clearwater, FL, USA."},{"key":"ref_35","unstructured":"Bezdek, J.C. (2006). Fuzzy Models and Algorithms for Pattern Recognition and Image Processing, Springer Science & Business Media."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1006\/dspr.1999.0361","article-title":"Speaker verification using adapted Gaussian mixture models","volume":"10","author":"Reynolds","year":"2000","journal-title":"Digit. Signal Process."},{"key":"ref_37","unstructured":"Vapnik, V. (2013). The Nature of Statistical Learning Theory, Springer Science & Business Media."},{"key":"ref_38","unstructured":"Auger, F., Patrick, F., Paulo, G., and Olivier, L. (1996). Time-Frequency Toolbox, CNRS France-Rice University."},{"key":"ref_39","unstructured":"Tong, M., and Minghao, T. (2010, January 23\u201325). LEACH-B: An improved LEACH protocol for wireless sensor network. Proceedings of the 6th International Conference on Wireless Communications Networking and Mobile Computing (WiCOM), Chengdu, China."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Rigamonti, R., Matthew, A.B., and Vincent, L. (2011, January 20\u201325). Are sparse representations really relevant for image classification?. Proceedings of the 2011 IEEE Computer Vision and Pattern Recognition (CVPR), Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995313"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Dehzangi, O., Zheng, Z., Mohammad-Mahdi, B., John, B., Christopher, R., and Roozbeh, J. (2013, January 1\u20133). The impact of vibrotactile biofeedback on the excessive walking sway and the postural control in elderly. Proceedings of the 4th Conference on Wireless Health, Baltimore, MD, USA.","DOI":"10.1145\/2534088.2534110"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/17\/12\/2735\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:51:27Z","timestamp":1760208687000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/17\/12\/2735"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,11,27]]},"references-count":41,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2017,12]]}},"alternative-id":["s17122735"],"URL":"https:\/\/doi.org\/10.3390\/s17122735","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,11,27]]}}}