{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,5]],"date-time":"2026-07-05T11:30:03Z","timestamp":1783251003281,"version":"3.54.6"},"reference-count":49,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2018,9,18]],"date-time":"2018-09-18T00:00:00Z","timestamp":1537228800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key Research and Development Project of China","award":["2017YFC0704100"],"award-info":[{"award-number":["2017YFC0704100"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Gesture recognition acts as a key enabler for user-friendly human-computer interfaces (HCI). To bridge the human-computer barrier, numerous efforts have been devoted to designing accurate fine-grained gesture recognition systems. Recent advances in wireless sensing hold promise for a ubiquitous, non-invasive and low-cost system with existing Wi-Fi infrastructures. In this paper, we propose DeepNum, which enables fine-grained finger gesture recognition with only a pair of commercial Wi-Fi devices. The key insight of DeepNum is to incorporate the quintessence of deep learning-based image processing so as to better depict the influence induced by subtle finger movements. In particular, we make multiple efforts to transfer sensitive Channel State Information (CSI) into depth radio images, including antenna selection, gesture segmentation and image construction, followed by noisy image purification using high-dimensional relations. To fulfill the restrictive size requirements of deep learning model, we propose a novel region-selection method to constrain the image size and select qualified regions with dominant color and texture features. Finally, a 7-layer Convolutional Neural Network (CNN) and SoftMax function are adopted to achieve automatic feature extraction and accurate gesture classification. Experimental results demonstrate the excellent performance of DeepNum, which recognizes 10 finger gestures with overall accuracy of 98% in three typical indoor scenarios.<\/jats:p>","DOI":"10.3390\/s18093142","type":"journal-article","created":{"date-parts":[[2018,9,18]],"date-time":"2018-09-18T11:52:29Z","timestamp":1537271549000},"page":"3142","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":28,"title":["From Signal to Image: Enabling Fine-Grained Gesture Recognition with Commercial Wi-Fi Devices"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5509-0902","authenticated-orcid":false,"given":"Qizhen","family":"Zhou","sequence":"first","affiliation":[{"name":"National Defense Engineering College, Army Engineering University of PLA, Nanjing 210007, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianchun","family":"Xing","sequence":"additional","affiliation":[{"name":"National Defense Engineering College, Army Engineering University of PLA, Nanjing 210007, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wei","family":"Chen","sequence":"additional","affiliation":[{"name":"National Defense Engineering College, Army Engineering University of PLA, Nanjing 210007, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xuewei","family":"Zhang","sequence":"additional","affiliation":[{"name":"National Defense Engineering College, Army Engineering University of PLA, Nanjing 210007, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qiliang","family":"Yang","sequence":"additional","affiliation":[{"name":"National Defense Engineering College, Army Engineering University of PLA, Nanjing 210007, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2018,9,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10462-012-9356-9","article-title":"Vision based hand gesture recognition for human computer interaction: A survey","volume":"43","author":"Rautaray","year":"2015","journal-title":"Artif. Intell. Rev."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Panwar, M., and Mehra, P.S. (2011, January 3\u20135). Hand gesture recognition for human computer interaction. Proceedings of the 2011 International Conference on Image Information Processing, Shimla, India.","DOI":"10.1109\/ICIIP.2011.6108940"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Xu, C., Pathak, P.H., and Mohapatra, P. (2015, January 12\u201313). Finger-writing with smartwatch: A case for finger and hand gesture recognition using smartwatch. Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications, Santa Fe, NM, USA.","DOI":"10.1145\/2699343.2699350"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"11842","DOI":"10.3390\/s130911842","article-title":"Human-computer interaction based on hand gestures using RGB-D sensors","volume":"13","author":"Palacios","year":"2013","journal-title":"Sensors"},{"key":"ref_5","first-page":"20","article-title":"Latent support vector machine modeling for sign language recognition with Kinect","volume":"6","author":"Sun","year":"2015","journal-title":"ACM Trans. Intell. Syst. Technol. (TIST)"},{"key":"ref_6","unstructured":"Funasaka, M., Ishikawa, Y., Takata, M., and Joe, K. (2015, January 27\u201330). Sign language recognition using leap motion controller. Proceedings of the International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA), Las Vegas, NV, USA."},{"key":"ref_7","unstructured":"Pu, Q., Gupta, S., Gollakota, S., and Patel, S. (October, January 30). Whole-home gesture recognition using wireless signals. Proceedings of the 19th annual international conference on Mobile computing & networking, Miami, FL, USA."},{"key":"ref_8","unstructured":"Kellogg, B., Talla, V., and Gollakota, S. (2014, January 2\u20134). Bringing gesture recognition to all devices. Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation, Seattle, WA, USA."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Wang, J., Vasisht, D., and Katabi, D. (2014, January 17\u201322). RF-IDraw: Virtual touch screen in the air using RF signals. Proceedings of the 2014 ACM Conference on SIGCOMM, Chicago, IL, USA.","DOI":"10.1145\/2619239.2626330"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Abdelnasser, H., Youssef, M., and Harras, K. (May, January 26). Wigest: A ubiquitous WiFi-based gesture recognition system. Proceedings of the 2015 IEEE Conference on Computer Communications (INFOCOM), Hong Kong, China.","DOI":"10.1109\/INFOCOM.2015.7218525"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Sun, L., Sen, S., Koutsonikolas, D., and Kim, K. (2015, January 7\u201311). Widraw: Enabling hands-free drawing in the air on commodity wifi devices. Proceedings of the 21st Annual International Conference on Mobile Computing and Networking, Paris, France.","DOI":"10.1145\/2789168.2790129"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Ali, K., Liu, A.X., Wang, W., and Shahzad, M. (2015, January 7\u201311). Keystroke recognition using Wi-Fi signals. Proceedings of the 21st Annual International Conference on Mobile Computing and Networking, Paris, France.","DOI":"10.1145\/2789168.2790109"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Li, H., Yang, W., Wang, J., Xu, Y., and Huang, L. (2016, January 12\u201316). WiFinger: Talk to your smart devices with finger-grained gesture. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Heidelberg, Germany.","DOI":"10.1145\/2971648.2971738"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Tan, S., and Yang, J. (2016, January 5\u20138). WiFinger: Leveraging commodity Wi-Fi for fine-grained finger gesture recognition. Proceedings of the 17th ACM International Symposium on Mobile Ad Hoc Networking and Computing, Paderborn, Germany.","DOI":"10.1145\/2942358.2942393"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Virmani, A., and Shahzad, M. (2017, January 19\u201323). Position and orientation agnostic gesture recognition usingWi-Fi. Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services, Niagara Falls, NY, USA.","DOI":"10.1145\/3081333.3081340"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1109\/CC.2017.8233654","article-title":"Deep learning for wireless physical layer: Opportunities and challenges","volume":"14","author":"Wang","year":"2017","journal-title":"China Commun."},{"key":"ref_17","first-page":"763","article-title":"CSI-based fingerprinting for indoor localization: A deep learning approach","volume":"66","author":"Wang","year":"2017","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1016\/j.neucom.2016.02.055","article-title":"Deep neural networks for wireless localization in indoor and outdoor environments","volume":"194","author":"Zhang","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Shi, C., Liu, J., Liu, H., and Chen, Y. (2017, January 10\u201314). Smart user authentication through actuation of daily activities leveraging WiFi-enabled IoT. Proceedings of the 18th ACM International Symposium on Mobile Ad Hoc Networking and Computing, Chennai, India.","DOI":"10.1145\/3084041.3084061"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"10346","DOI":"10.1109\/TVT.2017.2737553","article-title":"CSI-Based Device-Free Wireless Localization and Activity Recognition Using Radio Image Features","volume":"66","author":"Gao","year":"2017","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/2200000006","article-title":"Learning deep architectures for AI. Found","volume":"2","author":"Bengio","year":"2009","journal-title":"Trends Mach. Learn."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zhou, Q., Xing, J., Li, J., and Yang, Q. (2016, January 19). A device-free number gesture recognition approach based on deep learning. Proceedings of the 2016 12th International Conference on Computational Intelligence and Security (CIS), Wuxi, China.","DOI":"10.1109\/CIS.2016.0022"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1145\/3191755","article-title":"SignFi: Sign Language Recognition Using WiFi","volume":"2","author":"Ma","year":"2018","journal-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Molchanov, P., Gupta, S., Kim, K., and Pulli, K. (2015, January 4\u20138). Multi-sensor system for driver\u2019s hand-gesture recognition. Proceedings of the 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), Ljubljana, Slovenia.","DOI":"10.1109\/FG.2015.7163132"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"142","DOI":"10.1145\/2897824.2925953","article-title":"Soli: Ubiquitous gesture sensing with millimeter wave radar","volume":"35","author":"Lien","year":"2016","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1145\/1851275.1851203","article-title":"Predictable 802.11 packet delivery from wireless channel measurements","volume":"40","author":"Halperin","year":"2010","journal-title":"ACM SIGCOMM Comput. Commun. Rev."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Xie, Y., Li, Z., and Li, M. (2018). Precise power delay profiling with commodity Wi-Fi. IEEE Trans. Mob. Comput.","DOI":"10.1109\/TMC.2018.2860991"},{"key":"ref_28","unstructured":"Qian, K., Wu, C., Yang, Z., Liu, Y., and Jamieson, K. (2017, January 10\u201314). Widar: Decimeter-level passive tracking via velocity monitoring with commodity Wi-Fi. Proceedings of the 18th ACM International Symposium on Mobile Ad Hoc Networking and Computing, Chennai, India."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Qian, K., Wu, C., Zhang, Y., Zhang, G., Yang, Z., and Liu, Y. (2018, January 2\u20135). Widar 2.0: Passive human tracking with a single wi-fi link. Proceedings of the ACM MobiSys, Athen, Athens, Greece.","DOI":"10.1145\/3210240.3210314"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"511","DOI":"10.1109\/TMC.2016.2557795","article-title":"RT-Fall: A Real-Time and Contactless Fall Detection System with Commodity WiFi Devices","volume":"16","author":"Wang","year":"2017","journal-title":"IEEE Trans. Mob. Comput."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Wang, W., Liu, A.X., and Shahzad, M. (2016, January 12\u201316). Gait recognition using wifi signals. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Heidelberg, Germany.","DOI":"10.1145\/2971648.2971670"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Guo, X., Liu, B., Shi, C., Liu, H., Chen, Y., and Chuah, M. (2017, January 6\u20138). WiFi-Enabled Smart Human Dynamics Monitoring. Proceedings of the 15th ACM Conference on Embedded Network Sensor Systems, Delft, The Netherlands.","DOI":"10.1145\/3131672.3131692"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"504","DOI":"10.1126\/science.1127647","article-title":"Reducing the dimensionality of data with neural networks","volume":"313","author":"Hinton","year":"2006","journal-title":"Science"},{"key":"ref_34","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20136). Imagenet classification with deep convolutional neural networks. Proceedings of the Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1109\/TASL.2011.2134090","article-title":"Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition","volume":"20","author":"Dahl","year":"2012","journal-title":"IEEE Trans. Audio Speech Lang. Process."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Liu, C., Zhang, L., Liu, Z., Liu, K., Li, X., and Liu, Y. (2016, January 3\u20137). Lasagna: Towards deep hierarchical understanding and searching over mobile sensing data. Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking, NY, USA.","DOI":"10.1145\/2973750.2973752"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Wang, X., Wang, X., and Mao, S. (2017, January 21\u201325). CiFi: Deep convolutional neural networks for indoor localization with 5 GHz Wi-Fi. Proceedings of the 2017 IEEE International Conference on Communications (ICC), Paris, France.","DOI":"10.1109\/ICC.2017.7997235"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"789","DOI":"10.1109\/THMS.2017.2693242","article-title":"Qualitative action recognition by wireless radio signals in human\u2013machine systems","volume":"47","author":"Lv","year":"2017","journal-title":"IEEE Trans. Hum. Mach. Syst."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1904","DOI":"10.1109\/TPAMI.2015.2389824","article-title":"Spatial pyramid pooling in deep convolutional networks for visual recognition","volume":"37","author":"He","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intel."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Zheng, X., Wang, J., Shangguan, L., Zhou, Z., and Liu, Y. (2016, January 10\u201314). Smokey: Ubiquitous smoking detection with commercial wifi infrastructures. Proceedings of the IEEE INFOCOM 2016-The 35th Annual IEEE International Conference on Computer Communications, San Francisco, CA, USA.","DOI":"10.1109\/INFOCOM.2016.7524399"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Qian, K., Wu, C., Zhou, Z., Zheng, Y., Yang, Z., and Liu, Y. (2017, January 6\u201311). Inferring motion direction using commodity wi-fi for interactive exergames. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA.","DOI":"10.1145\/3025453.3025678"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"849","DOI":"10.1109\/TPAMI.2012.140","article-title":"Image denoising using the higher order singular value decomposition","volume":"35","author":"Rajwade","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_43","unstructured":"Brett, W., and Tamara, G. (2015, February 06). MATLAB Tensor Toolbox Version 2.6, Available online: http:\/\/www.sandia.gov\/~tgkolda\/TensorToolbox\/."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Girshick, R., Donahue, J., Darrell, T., and Malik, J. (2014, January 23). Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.81"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1007\/s11263-013-0620-5","article-title":"Selective search for object recognition","volume":"104","author":"Uijlings","year":"2013","journal-title":"Int. J. Comput. Vis."},{"key":"ref_46","first-page":"1556","article-title":"Very deep convolutional networks for large-scale image recognition","volume":"1409","author":"Simonyan","year":"2014","journal-title":"arXiv preprint arXiv"},{"key":"ref_47","first-page":"8","article-title":"TensorBeat: Tensor Decomposition for Monitoring Multi-Person Breathing Beats with Commodity WiFi","volume":"9","author":"Wang","year":"2017","journal-title":"ACM Trans. Intell. Syst. Technol."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/9\/3142\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:21:04Z","timestamp":1760196064000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/18\/9\/3142"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,9,18]]},"references-count":49,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2018,9]]}},"alternative-id":["s18093142"],"URL":"https:\/\/doi.org\/10.3390\/s18093142","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,9,18]]}}}