{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T15:46:51Z","timestamp":1774540011402,"version":"3.50.1"},"reference-count":30,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2024,11,9]],"date-time":"2024-11-09T00:00:00Z","timestamp":1731110400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62172080, 62271124"],"award-info":[{"award-number":["62172080, 62271124"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100018542","name":"Natural Science Foundation of Sichuan Province","doi-asserted-by":"crossref","award":["2023NSFSC0478, 2023NSFSC0488"],"award-info":[{"award-number":["2023NSFSC0478, 2023NSFSC0488"]}],"id":[{"id":"10.13039\/501100018542","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100012166","name":"National Key R&D Program of China","doi-asserted-by":"crossref","award":["2022YFB3103404"],"award-info":[{"award-number":["2022YFB3103404"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Priv. Secur."],"published-print":{"date-parts":[[2025,2,28]]},"abstract":"<jats:p>As one of the most common ways for user authentication, Personal Identification Number (PIN), due to its simplicity and convenience, has suffered from plenty of side-channel attacks, which pose a severe threat to people\u2019s privacy and property. The success of existing attacks is usually built upon the premise of no occlusion between the attacker and the victim\u2019s hand gesture, but it increases the difficulty of launching the attack and the possibility of exposure. To overcome such limitation, we propose ArmSpy++, an improved video-assisted PIN inference attack built upon our previous research, ArmSpy. Specifically, ArmSpy++ employs new modules to leverage more features like the keystroke-induced elbow bending, wrist speed variation, and the spatial relationship between different arm joints, to correctly detect Keystrokes. ArmSpy++ delves into the perspective relationship and natural typing habits to ensure a high success rate of PIN inference. We also re-designed the inferred PIN pattern coordination mechanism to accurately deduce the PINs. By using a pre-trained HigherHRNet model for posture estimation ArmSpy++ eliminates the necessity of additional training. The extensive experiments demonstrate that ArmSpy++ can achieve over 83.1% average accuracy with 3 attempts and even 92.5% for some victims, indicating the severity of the threat posed by ArmSpy++.<\/jats:p>","DOI":"10.1145\/3696418","type":"journal-article","created":{"date-parts":[[2024,9,23]],"date-time":"2024-09-23T10:50:29Z","timestamp":1727088629000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["ArmSpy++: Enhanced PIN Inference through Video-based Fine-grained Arm Posture Analysis"],"prefix":"10.1145","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2708-2790","authenticated-orcid":false,"given":"Huan","family":"Dai","sequence":"first","affiliation":[{"name":"Suzhou University of Science and Technology, Suzhou, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-5769-6469","authenticated-orcid":false,"given":"Yuefeng","family":"Chen","sequence":"additional","affiliation":[{"name":"University of Electronic Science and Technology of China, Chengdu, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5105-4648","authenticated-orcid":false,"given":"Yicong","family":"Du","sequence":"additional","affiliation":[{"name":"University of Electronic Science and Technology of China, Chengdu, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6712-314X","authenticated-orcid":false,"given":"Luping","family":"Wang","sequence":"additional","affiliation":[{"name":"Suzhou University of Science and Technology, Suzhou, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-6180-9961","authenticated-orcid":false,"given":"Ziyu","family":"Shao","sequence":"additional","affiliation":[{"name":"University of Electronic Science and Technology of China, Chengdu, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1162-839X","authenticated-orcid":false,"given":"Hongbo","family":"Liu","sequence":"additional","affiliation":[{"name":"University of Electronic Science and Technology of China, Chengdu, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2286-1384","authenticated-orcid":false,"given":"Yanzhi","family":"Ren","sequence":"additional","affiliation":[{"name":"University of Electronic Science and Technology of China, Chengdu, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0207-9643","authenticated-orcid":false,"given":"Jiadi","family":"Yu","sequence":"additional","affiliation":[{"name":"Shanghai Jiao Tong University, Shanghai, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2519-6196","authenticated-orcid":false,"given":"Bo","family":"Liu","sequence":"additional","affiliation":[{"name":"Auburn University, Auburn, United States"}]}],"member":"320","published-online":{"date-parts":[[2024,11,9]]},"reference":[{"key":"e_1_3_2_2_2","first-page":"90","volume-title":"Proceedings of the Annual International Conference on Mobile Computing and Networking","author":"Ali Kamran","year":"2015","unstructured":"Kamran Ali, Alex X Liu, Wei Wang, and Muhammad Shahzad. 2015. Keystroke recognition using WiFi signals. In Proceedings of the Annual International Conference on Mobile Computing and Networking. 90\u2013102."},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/SECPRI.2004.1301311"},{"issue":"2011","key":"e_1_3_2_4_2","first-page":"9","article-title":"TouchLogger: Inferring keystrokes on touch screen from smartphone motion","volume":"11","author":"Cai Liang","year":"2011","unstructured":"Liang Cai and Hao Chen. 2011. TouchLogger: Inferring keystrokes on touch screen from smartphone motion. USENIX Workshop on Hot Topics in Security 11, 2011 (2011), 9.","journal-title":"USENIX Workshop on Hot Topics in Security"},{"key":"e_1_3_2_5_2","first-page":"1687","volume-title":"Proceedings of the 31st USENIX Security Symposium","author":"Cardaioli Matteo","year":"2022","unstructured":"Matteo Cardaioli, Stefano Cecconello, Mauro Conti, Simone Milani, Stjepan Picek, and Eugen Saraci. 2022. Hand me your PIN! Inferring ATM PINs of users typing with a covered hand. In Proceedings of the 31st USENIX Security Symposium. 1687\u20131704."},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/INFOCOM48880.2022.9796738"},{"key":"e_1_3_2_7_2","first-page":"144","volume-title":"Proceedings of the IEEE Symposium on Security and Privacy","author":"Chen Yimin","year":"2018","unstructured":"Yimin Chen, Tao Li, Rui Zhang, Yanchao Zhang, and Terri Hedgpeth. 2018. EyeTell: Video-assisted touchscreen keystroke inference from eye movements. In Proceedings of the IEEE Symposium on Security and Privacy. IEEE, 144\u2013160."},{"key":"e_1_3_2_8_2","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Cheng Bowen","year":"2020","unstructured":"Bowen Cheng, Bin Xiao, Jingdong Wang, Honghui Shi, Thomas S. Huang, and Lei Zhang. 2020. HigherHRNet: Scale-aware representation learning for bottom-Up human pose estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition."},{"issue":"1","key":"e_1_3_2_9_2","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1007\/s10207-018-0403-7","article-title":"Differential audio analysis: A new side-channel attack on PIN pads","volume":"18","author":"Faria Gerson de Souza","year":"2019","unstructured":"Gerson de Souza Faria and Hae Yong Kim. 2019. Differential audio analysis: A new side-channel attack on PIN pads. International Journal of Information Security 18, 1 (2019), 73\u201384.","journal-title":"International Journal of Information Security"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/INFOCOM41043.2020.9155539"},{"key":"e_1_3_2_11_2","first-page":"1068","volume-title":"Proceedings of the ACM SIGSAC Conference on Computer and Communications Security","author":"Li Mengyuan","year":"2016","unstructured":"Mengyuan Li, Yan Meng, Junyi Liu, Haojin Zhu, Xiaohui Liang, Yao Liu, and Na Ruan. 2016. When CSI meets public WiFi: Inferring your mobile phone password via WiFi signals. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. 1068\u20131079."},{"key":"e_1_3_2_12_2","first-page":"217","volume-title":"Proceedings of the 2020 IEEE Symposium on Security and Privacy","author":"Li Zhengxiong","year":"2020","unstructured":"Zhengxiong Li, Fenglong Ma, Aditya Singh Rathore, Zhuolin Yang, Baicheng Chen, Lu Su, and Wenyao Xu. 2020. Wavespy: Remote and through-wall screen attack via mmwave sensing. In Proceedings of the 2020 IEEE Symposium on Security and Privacy. IEEE, 217\u2013232."},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/3433210.3453075"},{"key":"e_1_3_2_14_2","first-page":"1273","volume-title":"Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security","author":"Liu Xiangyu","year":"2015","unstructured":"Xiangyu Liu, Zhe Zhou, Wenrui Diao, Zhou Li, and Kehuan Zhang. 2015. When good becomes evil: Keystroke inference with smartwatch. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. 1273\u20131285."},{"key":"e_1_3_2_15_2","first-page":"27","volume-title":"Proceedings of the ACM International Symposium on Wearable Computers","author":"Maiti Anindya","year":"2015","unstructured":"Anindya Maiti, Murtuza Jadliwala, Jibo He, and Igor Bilogrevic. 2015. (Smart) watch your taps: Side-channel keystroke inference attacks using smartwatches. In Proceedings of the ACM International Symposium on Wearable Computers. 27\u201330."},{"key":"e_1_3_2_16_2","volume-title":"Handbook of Applied Cryptography","author":"Menezes Alfred J","year":"2018","unstructured":"Alfred J Menezes, Paul C Van Oorschot, and Scott A Vanstone. 2018. Handbook of Applied Cryptography. CRC press."},{"key":"e_1_3_2_17_2","first-page":"1","volume-title":"Proceedings of the Workshop on Mobile Computing Systems & Applications","author":"Owusu Emmanuel","year":"2012","unstructured":"Emmanuel Owusu, Jun Han, Sauvik Das, Adrian Perrig, and Joy Zhang. 2012. Accessory: Password inference using accelerometers on smartphones. In Proceedings of the Workshop on Mobile Computing Systems & Applications. 1\u20136."},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP40776.2020.9053420"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3211960.3211973"},{"key":"e_1_3_2_20_2","doi-asserted-by":"crossref","unstructured":"Mohd Sabra Anindya Maiti and Murtuza Jadliwala. 2021. Zoom on the Keystrokes: Exploiting video calls for keystroke inference attacks. In Proceedings of the Network and Distributed Systems Security Symposium.","DOI":"10.14722\/ndss.2021.23063"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1021\/ac60214a047"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/2906388.2906407"},{"key":"e_1_3_2_23_2","first-page":"904","volume-title":"Proceedings of the ACM SIGSAC Conference on Computer and Communications Security","author":"Shukla Diksha","year":"2014","unstructured":"Diksha Shukla, Rajesh Kumar, Abdul Serwadda, and Vir V Phoha. 2014. Beware, your hands reveal your secrets! In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. 904\u2013917."},{"key":"e_1_3_2_24_2","volume-title":"Proceedings of the Network and Distributed System Security Symposium","author":"Sun Jingchao","year":"2016","unstructured":"Jingchao Sun, Xiaocong Jin, Yimin Chen, Jinxue Zhang, Yanchao Zhang, and Rui Zhang. 2016. Visible: Video-assisted keystroke inference from tablet backside motion. In Proceedings of the Network and Distributed System Security Symposium."},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.5555\/938978.939190"},{"key":"e_1_3_2_26_2","unstructured":"Jaikumar Vijayan. 2007. TJX Data Breach: At 45.6M Card Numbers It\u2019s the Biggest Ever. Retrieved from https:\/\/www.computerworld.com\/article\/2544306\/tjx-data-breach--at-45-6m-card-numbers--it-s-the-biggest-ever.html"},{"key":"e_1_3_2_27_2","first-page":"1","volume-title":"Proceedings of the USENIX Security Symposium","volume":"8","author":"Vuagnoux Martin","year":"2009","unstructured":"Martin Vuagnoux and Sylvain Pasini. 2009. Compromising electromagnetic emanations of wired and wireless keyboards. In Proceedings of the USENIX Security Symposium, Vol. 8. 1\u201316."},{"key":"e_1_3_2_28_2","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1145\/2897845.2897847","volume-title":"Proceedings of the 11th ACM on Asia Conference on Computer and Communications Security","author":"Wang Chen","year":"2016","unstructured":"Chen Wang, Xiaonan Guo, Yan Wang, Yingying Chen, and Bo Liu. 2016. Friend or foe? Your wearable devices reveal your personal pin. In Proceedings of the 11th ACM on Asia Conference on Computer and Communications Security. 189\u2013200."},{"key":"e_1_3_2_29_2","first-page":"113","volume-title":"Proceedings of the ACM Conference on Security and Privacy in Wireless and Mobile Networks","author":"Xu Zhi","year":"2012","unstructured":"Zhi Xu, Kun Bai, and Sencun Zhu. 2012. Taplogger: Inferring user inputs on smartphone touchscreens using on-board motion sensors. In Proceedings of the ACM Conference on Security and Privacy in Wireless and Mobile Networks. 113\u2013124."},{"key":"e_1_3_2_30_2","first-page":"1403","volume-title":"Proceedings of the ACM SIGSAC Conference on Computer and Communications Security","author":"Yue Qinggang","year":"2014","unstructured":"Qinggang Yue, Zhen Ling, Xinwen Fu, Benyuan Liu, Kui Ren, and Wei Zhao. 2014. Blind recognition of touched keys on mobile devices. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security. 1403\u20131414."},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/INFOCOM.2018.8486006"}],"container-title":["ACM Transactions on Privacy and Security"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3696418","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3696418","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:18:55Z","timestamp":1750295935000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3696418"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,9]]},"references-count":30,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,2,28]]}},"alternative-id":["10.1145\/3696418"],"URL":"https:\/\/doi.org\/10.1145\/3696418","relation":{},"ISSN":["2471-2566","2471-2574"],"issn-type":[{"value":"2471-2566","type":"print"},{"value":"2471-2574","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,9]]},"assertion":[{"value":"2023-09-06","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-09-03","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-11-09","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}