{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T11:43:24Z","timestamp":1778154204275,"version":"3.51.4"},"reference-count":52,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2020,5,31]],"date-time":"2020-05-31T00:00:00Z","timestamp":1590883200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2020,8,31]]},"abstract":"<jats:p>Biometric authentication involves various technologies to identify individuals by exploiting their unique, measurable physiological and behavioral characteristics. However, traditional biometric authentication systems (e.g., face recognition, iris, retina, voice, and fingerprint) are at increasing risks of being tricked by biometric tools such as anti-surveillance masks, contact lenses, vocoder, or fingerprint films. In this article, we design a multimodal biometric authentication system named DeepKey, which uses both Electroencephalography (EEG) and gait signals to better protect against such risk. DeepKey consists of two key components: an Invalid ID Filter Model to block unauthorized subjects, and an identification model based on attention-based Recurrent Neural Network (RNN) to identify a subject\u2019s EEG IDs and gait IDs in parallel. The subject can only be granted access while all the components produce consistent affirmations to match the user\u2019s proclaimed identity. We implement DeepKey with a live deployment in our university and conduct extensive empirical experiments to study its technical feasibility in practice. DeepKey achieves the False Acceptance Rate (FAR) and the False Rejection Rate (FRR) of 0 and 1.0%, respectively. The preliminary results demonstrate that DeepKey is feasible, shows consistent superior performance compared to a set of methods, and has the potential to be applied to the authentication deployment in real-world settings.<\/jats:p>","DOI":"10.1145\/3393619","type":"journal-article","created":{"date-parts":[[2020,6,1]],"date-time":"2020-06-01T04:30:26Z","timestamp":1590985826000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":35,"title":["DeepKey"],"prefix":"10.1145","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5097-2113","authenticated-orcid":false,"given":"Xiang","family":"Zhang","sequence":"first","affiliation":[{"name":"University of New South Wales, Sydney, NSW, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4149-839X","authenticated-orcid":false,"given":"Lina","family":"Yao","sequence":"additional","affiliation":[{"name":"University of New South Wales, Sydney, NSW, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chaoran","family":"Huang","sequence":"additional","affiliation":[{"name":"University of New South Wales, Sydney, NSW, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tao","family":"Gu","sequence":"additional","affiliation":[{"name":"RMIT University, Melbourne, VIC, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zheng","family":"Yang","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yunhao","family":"Liu","sequence":"additional","affiliation":[{"name":"Michigan State University, East Lansing, MI, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2020,5,31]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.23919\/BIOSIG.2017.8053523"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-26561-2_20"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/NER.2011.5910581"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/IECBES.2016.7843496"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2013.2266578"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1093\/ageing\/afp017"},{"key":"e_1_2_1_7_1","volume-title":"Shazeer","author":"Chan William","year":"2017","unstructured":"William Chan , Navdeep Jaitly , Quoc V. Le , Oriol Vinyals , and Noam M . Shazeer . 2017 . Speech recognition with attention-based recurrent neural networks. U.S. Patent No . 9,799,327 [P]. 2017-10-24. William Chan, Navdeep Jaitly, Quoc V. Le, Oriol Vinyals, and Noam M. Shazeer. 2017. Speech recognition with attention-based recurrent neural networks. U.S. Patent No. 9,799,327 [P]. 2017-10-24."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-41320-9_1"},{"key":"e_1_2_1_9_1","unstructured":"Guglielmo Cola Marco Avvenuti Fabio Musso and Alessio Vecchio. 2016. Gait-based authentication using a wrist-worn device. In Mobiquitous. ACM 208--217.  Guglielmo Cola Marco Avvenuti Fabio Musso and Alessio Vecchio. 2016. Gait-based authentication using a wrist-worn device. In Mobiquitous. ACM 208--217."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.244676"},{"key":"e_1_2_1_11_1","volume-title":"International Conference of the Biometrics Special Interest Group (BIOSIG\u201914)","author":"Derawi Mohammad","year":"2014","unstructured":"Mohammad Derawi and Iurii Voitenko . 2014 . Fusion of gait and ECG for biometric user authentication . In International Conference of the Biometrics Special Interest Group (BIOSIG\u201914) . IEEE, 1--4. Mohammad Derawi and Iurii Voitenko. 2014. Fusion of gait and ECG for biometric user authentication. In International Conference of the Biometrics Special Interest Group (BIOSIG\u201914). IEEE, 1--4."},{"key":"e_1_2_1_12_1","volume-title":"2019 International Joint Conference on Neural Networks (IJCNN\u201919)","author":"Dong Manqing","unstructured":"Manqing Dong , Lina Yao , Xianzhi Wang , Boualem Benatallah , Xiang Zhang , and Quan Z. Sheng . 2019. Dual-stream self-attentive random forest for false information detection . In 2019 International Joint Conference on Neural Networks (IJCNN\u201919) . IEEE, 1--8. Manqing Dong, Lina Yao, Xianzhi Wang, Boualem Benatallah, Xiang Zhang, and Quan Z. Sheng. 2019. Dual-stream self-attentive random forest for false information detection. In 2019 International Joint Conference on Neural Networks (IJCNN\u201919). IEEE, 1--8."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2013.48"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1002\/wics.1262"},{"key":"e_1_2_1_15_1","unstructured":"Steven Goldstein. 2016. Methods and systems for voice authentication service leveraging networking. U.S. Patent No. 9 282 096 [P]. 2016-3-8.  Steven Goldstein. 2016. Methods and systems for voice authentication service leveraging networking. U.S. Patent No. 9 282 096 [P]. 2016-3-8."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/SPMB.2014.7002950"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/TITB.2012.2207729"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1155\/2014\/438254"},{"key":"e_1_2_1_19_1","volume-title":"Electronics and Mobile Communication Conference (IEMCON\u201916)","author":"Jayarathne Isuru","year":"2016","unstructured":"Isuru Jayarathne , Michael Cohen , and Senaka Amarakeerthi . 2016 . BrainID: Development of an EEG-based biometric authentication system. In 2016 IEEE 7th Annual Information Technology , Electronics and Mobile Communication Conference (IEMCON\u201916) . IEEE, 1--6. Isuru Jayarathne, Michael Cohen, and Senaka Amarakeerthi. 2016. BrainID: Development of an EEG-based biometric authentication system. In 2016 IEEE 7th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON\u201916). IEEE, 1--6."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/IranianCEE.2016.7585697"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/IPTA.2016.7820954"},{"key":"e_1_2_1_22_1","volume-title":"Kingma and Jimmy Ba","author":"Diederik","year":"2014","unstructured":"Diederik P. Kingma and Jimmy Ba . 2014 . Adam : A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014). Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)."},{"key":"e_1_2_1_23_1","volume-title":"International Workshop on Informatics","author":"Konno Shinsuke","year":"2015","unstructured":"Shinsuke Konno , Yoshitaka Nakamura , Yoh Shiraishi , and Osamu Takahashi . 2015 . Gait-based authentication using trouser front-pocket sensors . International Workshop on Informatics (2015). Shinsuke Konno, Yoshitaka Nakamura, Yoh Shiraishi, and Osamu Takahashi. 2015. Gait-based authentication using trouser front-pocket sensors. International Workshop on Informatics (2015)."},{"key":"e_1_2_1_24_1","volume-title":"M. V. Ramakrishna, and B. D. C. N. Prasad.","author":"Kumar Pavan K.","year":"2013","unstructured":"Pavan K. Kumar , PESN Krishna Prasad , M. V. Ramakrishna, and B. D. C. N. Prasad. 2013 . Feature extraction using sparse SVD for biometric fusion in multimodal authentication. International Journal of Network Security 8 Its Applications 5, 4 (2013), 83. Pavan K. Kumar, PESN Krishna Prasad, M. V. Ramakrishna, and B. D. C. N. Prasad. 2013. Feature extraction using sparse SVD for biometric fusion in multimodal authentication. International Journal of Network Security 8 Its Applications 5, 4 (2013), 83."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2013.2294552"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIFS.2013.2252342"},{"key":"e_1_2_1_27_1","volume-title":"Latman and Emily Herb","author":"Neal","year":"2013","unstructured":"Neal S. Latman and Emily Herb . 2013 . A field study of the accuracy and reliability of a biometric iris recognition system. Science 8 Justice 53, 2 (2013), 98--102. Neal S. Latman and Emily Herb. 2013. A field study of the accuracy and reliability of a biometric iris recognition system. Science 8 Justice 53, 2 (2013), 98--102."},{"key":"e_1_2_1_28_1","volume-title":"Neural Networks: Tricks of the Trade","author":"LeCun Yann","unstructured":"Yann LeCun , L\u00e9on Bottou , Genevieve B. Orr , and Klaus-Robert M\u00fcller . 1998. Efficient backprop . In Neural Networks: Tricks of the Trade . Springer , 9--50. Yann LeCun, L\u00e9on Bottou, Genevieve B. Orr, and Klaus-Robert M\u00fcller. 1998. Efficient backprop. In Neural Networks: Tricks of the Trade. Springer, 9--50."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-32695-0_54"},{"key":"e_1_2_1_30_1","volume-title":"FVC2002: Second fingerprint verification competition. In Proceedings of the 16th International Conference on Pattern Recognition","volume":"3","author":"Maio Dario","year":"2002","unstructured":"Dario Maio , Davide Maltoni , Raffaele Cappelli , James L. Wayman , and Anil K. Jain . 2002 . FVC2002: Second fingerprint verification competition. In Proceedings of the 16th International Conference on Pattern Recognition , 2002 , Vol. 3 . IEEE, 811--814. Dario Maio, Davide Maltoni, Raffaele Cappelli, James L. Wayman, and Anil K. Jain. 2002. FVC2002: Second fingerprint verification competition. In Proceedings of the 16th International Conference on Pattern Recognition, 2002, Vol. 3. IEEE, 811--814."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2006.38"},{"key":"e_1_2_1_32_1","article-title":"Multimodal biometric authentication using ECG and fingerprint","volume":"111","author":"Manjunathswamy B. E.","year":"2015","unstructured":"B. E. Manjunathswamy , Appaji M. Abhishek , J. Thriveni , K. R. Venugopal , and L. M. Patnaik . 2015 . Multimodal biometric authentication using ECG and fingerprint . International Journal of Computer Applications 111 , 13 (2015). B. E. Manjunathswamy, Appaji M. Abhishek, J. Thriveni, K. R. Venugopal, and L. M. Patnaik. 2015. Multimodal biometric authentication using ECG and fingerprint. International Journal of Computer Applications 111, 13 (2015).","journal-title":"International Journal of Computer Applications"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2007.1012"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2014.2371335"},{"key":"e_1_2_1_35_1","volume-title":"Encyclopedia of Biometrics","author":"Yam C. Y.","unstructured":"C. Y. Yam and Mark Nixon . 2009. Model-based gait recognition . In Encyclopedia of Biometrics . Springer , 633--639. C. Y. Yam and Mark Nixon. 2009. Model-based gait recognition. In Encyclopedia of Biometrics. Springer, 633--639."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2013.98"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMC.2017.2778155"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2016.09.365"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.3390\/s140917037"},{"key":"e_1_2_1_40_1","volume-title":"Siddiqui","author":"Telgad Rupali L.","year":"2014","unstructured":"Rupali L. Telgad , P. D. Deshmukh , and Almas M. N . Siddiqui . 2014 . Combination approach to score level fusion for Multimodal Biometric system by using face and fingerprint. In Recent Advances and Innovations in Engineering (ICRAIE), 2014. IEEE , 1--8. Rupali L. Telgad, P. D. Deshmukh, and Almas M. N. Siddiqui. 2014. Combination approach to score level fusion for Multimodal Biometric system by using face and fingerprint. In Recent Advances and Innovations in Engineering (ICRAIE), 2014. IEEE, 1--8."},{"key":"e_1_2_1_41_1","volume-title":"IEEE International Conference on Systems, Man, and Cybernetics (SMC\u201916)","author":"Thomas Kavitha P.","unstructured":"Kavitha P. Thomas and A. P. Vinod . 2016. Utilizing individual alpha frequency and delta band power in EEG based biometric recognition . In IEEE International Conference on Systems, Man, and Cybernetics (SMC\u201916) . IEEE, 004787--004791. Kavitha P. Thomas and A. P. Vinod. 2016. Utilizing individual alpha frequency and delta band power in EEG based biometric recognition. In IEEE International Conference on Systems, Man, and Cybernetics (SMC\u201916). IEEE, 004787--004791."},{"key":"e_1_2_1_42_1","doi-asserted-by":"crossref","unstructured":"Kavitha P. Thomas and A. P. Vinod. 2017. EEG-based biometric authentication using gamma band power during rest state. Circuits Systems and Signal Processing (2017) 1--13.  Kavitha P. Thomas and A. P. Vinod. 2017. EEG-based biometric authentication using gamma band power during rest state. Circuits Systems and Signal Processing (2017) 1--13.","DOI":"10.1007\/s00034-017-0551-4"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2014.01.016"},{"key":"e_1_2_1_44_1","volume-title":"et\u00a0al","author":"Wang Yequan","year":"2016","unstructured":"Yequan Wang , Minlie Huang , Li Zhao , et\u00a0al . 2016 . Attention-based LSTM for aspect-level sentiment classification. In EMNLP. 606--615. Yequan Wang, Minlie Huang, Li Zhao, et\u00a0al. 2016. Attention-based LSTM for aspect-level sentiment classification. In EMNLP. 606--615."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2016.2545669"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMC.2018.2828816"},{"key":"e_1_2_1_47_1","volume-title":"21st International Conference on Pattern Recognition (ICPR\u201912)","author":"Yano Vitor","year":"2012","unstructured":"Vitor Yano , Alessandro Zimmer , and Lee Luan Ling . 2012 . Multimodal biometric authentication based on iris pattern and pupil light reflex . In 21st International Conference on Pattern Recognition (ICPR\u201912) . IEEE, 2857--2860. Vitor Yano, Alessandro Zimmer, and Lee Luan Ling. 2012. Multimodal biometric authentication based on iris pattern and pupil light reflex. In 21st International Conference on Pattern Recognition (ICPR\u201912). IEEE, 2857--2860."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMC.2017.2706282"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2015.102"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3264959"},{"key":"e_1_2_1_51_1","volume-title":"A survey on deep learning based brain computer interface: Recent advances and new frontiers. arXiv preprint arXiv:1905.04149","author":"Zhang Xiang","year":"2019","unstructured":"Xiang Zhang , Lina Yao , Xianzhi Wang , Jessica Monaghan , and David Mcalpine . 2019. A survey on deep learning based brain computer interface: Recent advances and new frontiers. arXiv preprint arXiv:1905.04149 ( 2019 ). Xiang Zhang, Lina Yao, Xianzhi Wang, Jessica Monaghan, and David Mcalpine. 2019. A survey on deep learning based brain computer interface: Recent advances and new frontiers. arXiv preprint arXiv:1905.04149 (2019)."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/THMS.2014.2340578"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3393619","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3393619","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:41:17Z","timestamp":1750200077000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3393619"}},"subtitle":["A Multimodal Biometric Authentication System via Deep Decoding Gaits and Brainwaves"],"short-title":[],"issued":{"date-parts":[[2020,5,31]]},"references-count":52,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2020,8,31]]}},"alternative-id":["10.1145\/3393619"],"URL":"https:\/\/doi.org\/10.1145\/3393619","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,5,31]]},"assertion":[{"value":"2019-06-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-04-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-05-31","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}