{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,29]],"date-time":"2026-05-29T23:06:01Z","timestamp":1780095961060,"version":"3.54.0"},"reference-count":33,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2021,4,23]],"date-time":"2021-04-23T00:00:00Z","timestamp":1619136000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["NRF2018X1A3A1070163"],"award-info":[{"award-number":["NRF2018X1A3A1070163"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With the ubiquity of wearable devices, various behavioural biometrics have been exploited for continuous user authentication during daily activities. However, biometric authentication using complex hand behaviours have not been sufficiently investigated. This paper presents an implicit and continuous user authentication model based on hand-object manipulation behaviour, using a finger-and hand-mounted inertial measurement unit (IMU)-based system and state-of-the-art deep learning models. We employed three convolutional neural network (CNN)-based deep residual networks (ResNets) with multiple depths (i.e., 50, 101, and 152 layers) and two recurrent neural network (RNN)-based long short-term memory (LSTMs): simple and bidirectional. To increase ecological validity, data collection of hand-object manipulation behaviours was based on three different age groups and simple and complex daily object manipulation scenarios. As a result, both the ResNets and LSTMs models acceptably identified users\u2019 hand behaviour patterns, with the best average accuracy of 96.31% and F1-score of 88.08%. Specifically, in the simple hand behaviour authentication scenarios, more layers in residual networks tended to show better performance without showing conventional degradation problems (the ResNet-152 &gt; ResNet-101 &gt; ResNet-50). In a complex hand behaviour scenario, the ResNet models outperformed user authentication compared to the LSTMs. The 152-layered ResNet and bidirectional LSTM showed an average false rejection rate of 8.34% and 16.67% and an equal error rate of 1.62% and 9.95%, respectively.<\/jats:p>","DOI":"10.3390\/s21092981","type":"journal-article","created":{"date-parts":[[2021,4,25]],"date-time":"2021-04-25T02:12:57Z","timestamp":1619316777000},"page":"2981","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Deep Residual Networks for User Authentication via Hand-Object Manipulations"],"prefix":"10.3390","volume":"21","author":[{"given":"Kanghae","family":"Choi","sequence":"first","affiliation":[{"name":"ImagineX Lab, Graduate School of Technology and Innovation Management, Hanyang University, Seoul 04763, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hokyoung","family":"Ryu","sequence":"additional","affiliation":[{"name":"ImagineX Lab, Graduate School of Technology and Innovation Management, Hanyang University, Seoul 04763, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8656-6387","authenticated-orcid":false,"given":"Jieun","family":"Kim","sequence":"additional","affiliation":[{"name":"ImagineX Lab, Graduate School of Technology and Innovation Management, Hanyang University, Seoul 04763, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,4,23]]},"reference":[{"key":"ref_1","unstructured":"Tiefenau, C., H\u00e4ring, M., Khamis, M., and von Zezschwitz, E. (2019). Please enter your PIN\u2014On the Risk of Bypass Attacks on Biometric Authentication on Mobile Devices. arXiv."},{"key":"ref_2","unstructured":"Khan, H., Hengartner, U., and Vogel, D. (2015, January 22\u201324). Usability and security perceptions of implicit authentication: Convenient, secure, some-times annoying. In Eleventh Symposium on Usable Privacy and Security. Presented at 11th Symposium on Usable Privacy and Security, Ottawa, ON, Canada."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Zhu, T., Weng, Z., Chen, G., and Fu, L. (2020). A Hybrid Deep Learning System for Real-World Mobile User Authentication Using Motion Sensors. Sensors, 20.","DOI":"10.3390\/s20143876"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"5008","DOI":"10.1109\/JIOT.2020.2975779","article-title":"AUToSen: Deep-Learning-Based Implicit Continuous Authentication Using Smartphone Sensors","volume":"7","author":"Abuhamad","year":"2020","journal-title":"IEEE Internet Things J."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Lee, W., and Lee, R. (2016, January 18). Implicit Sensor-Based Authentication of Smartphone Users with Smartwatch. Proceedings of the Hardware and Architectural Support for Security and Privacy 2016, Seoul, Korea.","DOI":"10.1145\/2948618.2948627"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Hong, F., Wei, M., You, S., Feng, Y., and Guo, Z. (2015, January 18\u201323). Waving authentication: Your smartphone authenticate you on motion gesture. Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems, Seoul, Korea.","DOI":"10.1145\/2702613.2725444"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Lee, W., Liu, X., Shen, Y., Jin, H., and Lee, R.B. (2017, January 21\u201323). Secure Pick Up: Implicit Authentication When You Start Using the Smartphone. Proceedings of the 22nd ACM on Symposium on Access Control Models and Technologies-SACMAT \u201817 Abstracts, Indianapolis, IN, USA.","DOI":"10.1145\/3078861.3078870"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Choi, M., Lee, S., Jo, M., and Shin, J. (2021). Keystroke Dynamics-Based Authentication Using Unique Keypad. Sensors, 21.","DOI":"10.3390\/s21062242"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"4417","DOI":"10.1007\/s12652-018-1123-6","article-title":"A continuous smartphone authentication method based on gait patterns and keystroke dynamics","volume":"10","author":"Lamiche","year":"2018","journal-title":"J. Ambient. Intell. Humaniz. Comput."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Sellahewa, H., Ibrahim, N., and Zeadally, S. (2019). Biometric Authentication for Wearables. Biometric-Based Physical and Cybersecurity Systems, Springer.","DOI":"10.1007\/978-3-319-98734-7_14"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Moon, J., Minaya, N.H., Le, N.A., Park, H.-C., and Choi, S.-I. (2020). Can Ensemble Deep Learning Identify People by Their Gait Using Data Collected from Multi-Modal Sensors in Their Insole?. Sensors, 20.","DOI":"10.3390\/s20144001"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Le, H.V., Mayer, S., Wolf, K., and Henze, N. (2016, January 7\u201312). Finger Placement and Hand Grasp during Smartphone Interaction. Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, San Jose, CA, USA.","DOI":"10.1145\/2851581.2892462"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"457","DOI":"10.1080\/07421222.2020.1759961","article-title":"Human Identification for Activities of Daily Living: A Deep Transfer Learning Approach","volume":"37","author":"Zhu","year":"2020","journal-title":"J. Manag. Inf. Syst."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Kontson, K., Marcus, I., Myklebust, B., and Civillico, E. (2017). Targeted box and blocks test: Normative data and comparison to standard tests. PLoS ONE, 12.","DOI":"10.1371\/journal.pone.0177965"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"3278","DOI":"10.1109\/JSEN.2018.2808688","article-title":"Latern: Dynamic Continuous Hand Gesture Recognition Using FMCW Radar Sensor","volume":"18","author":"Zhang","year":"2018","journal-title":"IEEE Sens. J."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1016\/j.procs.2019.08.027","article-title":"Towards Continuous Authentication on Mobile Phones using Deep Learning Models","volume":"155","author":"Volaka","year":"2019","journal-title":"Procedia Comput. Sci."},{"key":"ref_17","first-page":"1174","article-title":"Machine learning algorithms: A review","volume":"7","author":"Dey","year":"2016","journal-title":"Int. J. Comput. Sci. Inf. Technol. Adv. Res."},{"key":"ref_18","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1798","DOI":"10.1109\/TPAMI.2013.50","article-title":"Representation Learning: A Review and New Perspectives","volume":"35","author":"Bengio","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Shila, D.M., and Eyisi, E. (2018, January 1\u20133). Adversarial gait detection on mobile devices using recurrent neural networks. Proceedings of the 17th IEEE International Conference on Trust, Security and Privacy in Computing and Communications, New York, NY, USA.","DOI":"10.1109\/TrustCom\/BigDataSE.2018.00055"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Debard, Q., Wolf, C., Canu, S., and Arn\u00e9, J. (2018, January 15\u201319). Learning to recognize touch gestures: Recurrent vs. convolutional features and dynamic sampling. Proceedings of the 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi\u2019an, China.","DOI":"10.1109\/FG.2018.00026"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Kim, Y. (2014). Convolutional neural networks for sentence classification. arXiv.","DOI":"10.3115\/v1\/D14-1181"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Mekruksavanich, S., and Jitpattanakul, A. (2021). Biometric User Identification Based on Human Activity Recognition Using Wearable Sensors: An Experiment Using Deep Learning Models. Electronics, 10.","DOI":"10.3390\/electronics10030308"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1109\/72.279181","article-title":"Learning long-term dependencies with gradient descent is difficult","volume":"5","author":"Bengio","year":"1994","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Liu, J., Shahroudy, A., Xu, D., and Wang, G. (2016, January 11\u201314). Spatio-temporal lstm with trust gates for 3D human action recognition. Proceedings of the 14th European Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46487-9_50"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Lee, S., Lee, H., Lee, J., Ryu, H., Kim, I.Y., and Kim, J. (2020). Clip-On IMU System for Assessing Age-Related Changes in Hand Functions. Sensors, 20.","DOI":"10.3390\/s20216313"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Madgwick, S.O.H., Andrew, J.L.H., and Ravi, V. (July, January 29). Estimation of IMU and MARG orientation using a gradient descent algorithm. Proceedings of the 2011 IEEE International Conference on reh Abilitation Robotics, Zurich, Switzerland.","DOI":"10.1109\/ICORR.2011.5975346"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"320","DOI":"10.1016\/j.gaitpost.2011.05.018","article-title":"Quantification of inertial sensor-based 3D joint angle measurement accuracy using an instrumented gimbal","volume":"34","author":"Brennan","year":"2011","journal-title":"Gait Posture"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1016\/j.ipm.2009.03.002","article-title":"A systematic analysis of performance measures for classification tasks","volume":"45","author":"Sokolova","year":"2009","journal-title":"Inf. Process. Manag."},{"key":"ref_31","unstructured":"Yin, W., Kann, K., Yu, M., and Sch\u00fctze, H. (2017). Comparative study of cnn and rnn for natural language processing. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"125210","DOI":"10.1109\/ACCESS.2020.3005161","article-title":"Small and slim deep convolutional neural network for mobile device","volume":"8","author":"Winoto","year":"2020","journal-title":"IEEE Access"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Ranjan, J., and Whitehouse, K. (2015, January 7\u201311). Object hallmarks: Identifying object users using wearable wrist sensors. Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Osaka, Japan.","DOI":"10.1145\/2750858.2804263"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/9\/2981\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:52:11Z","timestamp":1760161931000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/9\/2981"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,23]]},"references-count":33,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2021,5]]}},"alternative-id":["s21092981"],"URL":"https:\/\/doi.org\/10.3390\/s21092981","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,23]]}}}