{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T18:10:14Z","timestamp":1782497414549,"version":"3.54.5"},"reference-count":41,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2019,5,31]],"date-time":"2019-05-31T00:00:00Z","timestamp":1559260800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100005047","name":"Natural Science Foundation of Liaoning Province","doi-asserted-by":"publisher","award":["20180520007"],"award-info":[{"award-number":["20180520007"]}],"id":[{"id":"10.13039\/501100005047","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities of China","doi-asserted-by":"publisher","award":["N172608005, N182612002, N182608003"],"award-info":[{"award-number":["N172608005, N182612002, N182608003"]}],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>For online sign language recognition (SLR) based on inertial measurement unit (IMU) and a surface electromyography (sEMG) sensor, achieving high-accuracy is a major challenge. The traditional method for that is the segmentation\u2013recognition mechanism, which has two key challenges: (1) it is difficult to design a highly robust segmentation method for online data with inconspicuous segmentation information; and (2) the diversity of input data will increase the burden of the classification. The recognition\u2013verification mechanism was proposed to improve the performance of online SLR. In the recognition stage, we used sliding windows to pull the data, and applied a convolutional neural network (CNN) to classify the sign language signal. In the verification stage, the confidence was evaluated by the Siamese network to judge the correctness of the classification results. The accuracy and rapidity of the classification model were discussed for 86 categories of Chinese sign language. In the experiments for online SLR, the superiority of the recognition\u2013verification mechanism compared to the segmentation\u2013recognition mechanism was verified.<\/jats:p>","DOI":"10.3390\/s19112495","type":"journal-article","created":{"date-parts":[[2019,5,31]],"date-time":"2019-05-31T11:59:56Z","timestamp":1559303996000},"page":"2495","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["An Recognition\u2013Verification Mechanism for Real-Time Chinese Sign Language Recognition Based on Multi-Information Fusion"],"prefix":"10.3390","volume":"19","author":[{"given":"Fei","family":"Wang","sequence":"first","affiliation":[{"name":"Faculty of Robot Science and Engineering, Northeastern University, Shenyang 110819, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shusen","family":"Zhao","sequence":"additional","affiliation":[{"name":"Faculty of Robot Science and Engineering, Northeastern University, Shenyang 110819, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xingqun","family":"Zhou","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Northeastern University, Shenyang 110004, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chen","family":"Li","sequence":"additional","affiliation":[{"name":"Faculty of Robot Science and Engineering, Northeastern University, Shenyang 110819, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mingyao","family":"Li","sequence":"additional","affiliation":[{"name":"Faculty of Robot Science and Engineering, Northeastern University, Shenyang 110819, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhen","family":"Zeng","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Northeastern University, Shenyang 110004, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2019,5,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Wei, S., Chen, X., Yang, X., Cao, S., and Zhang, X. (2016). A Component-Based Vocabulary-Extensible Sign Language Gesture Recognition Framework. Sensors, 16.","DOI":"10.3390\/s16040556"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"23303","DOI":"10.3390\/s150923303","article-title":"A Novel Phonology- and Radical-Coded Chinese Sign Language Recognition Framework Using Accelerometer and Surface Electromyography Sensors","volume":"15","author":"Cheng","year":"2015","journal-title":"Sensors"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"994","DOI":"10.1109\/JBHI.2016.2560907","article-title":"Chinese Sign Language Recognition Based on An Optimized Tree-structure Framework","volume":"21","author":"Yang","year":"2017","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1109\/THMS.2014.2302794","article-title":"A Hand Gesture Recognition Framework and Wearable Gesture-Based Interaction Prototype for Mobile Devices","volume":"44","author":"Lu","year":"2014","journal-title":"IEEE Trans. Hum. Mach. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Huang, J., Zhou, W., Zhang, Q., and Li, H. (2018, January 2\u20137). Video-based Sign Language Recognition without Temporal Segmentation. Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI-18), New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11903"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Quivira, F., Koike-Akino, T., Wang, Y., and Erdogmus, D. (2018, January 9). Translating sEMG Signals to Continuous Hand Poses using Recurrent Neural Networks. Proceedings of the IEEE Conference on Biomedical and Health Informatics (BHI), Las Vegas, NV, USA.","DOI":"10.1109\/BHI.2018.8333395"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1186\/s12938-018-0539-8","article-title":"PCA and deep learning based myoelectric grasping control of a prosthetic hand","volume":"17","author":"Li","year":"2018","journal-title":"Biomed. Eng. Online"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"815","DOI":"10.1016\/j.compeleceng.2018.02.041","article-title":"Service robot system with integration of wearable myo armband for specialized hand gesture human\u2013computer interfaces for people with disabilities with mobility problems","volume":"69","author":"Ding","year":"2018","journal-title":"Comput. Electr. Eng."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2695","DOI":"10.1109\/TBME.2012.2190734","article-title":"A Sign-Component-Based Framework for Chinese Sign Language Recognition Using Accelerometer and sEMG Data","volume":"59","author":"Li","year":"2012","journal-title":"IEEE Trans Biomed Eng."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Zhang, X., Chen, X., Wang, W.H., Yang, J., Lantz, V., and Wang, K. (2009, January 8\u201311). Hand gesture recognition and virtual game control based on 3D accelerometer and EMG sensors. Proceedings of the 2009 International Conference on Intelligent User Interfaces, Sanibel Island, FL, USA.","DOI":"10.1145\/1502650.1502708"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1614","DOI":"10.1016\/j.patcog.2010.12.014","article-title":"Model-based segmentation and recognition of dynamic gestures in continuous video streams","volume":"44","author":"Li","year":"2011","journal-title":"Pattern Recognit."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1016\/j.patcog.2018.02.011","article-title":"Structured Dynamic Time Warping for Continuous Hand Trajectory Gesture Recognition","volume":"80","author":"Tang","year":"2018","journal-title":"Pattern Recognit."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Cui, R., Liu, H., and Zhang, C. (2017, January 21\u201326). Recurrent Convolutional Neural Networks for Continuous Sign Language Recognition by Staged Optimization. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.175"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Camgoz, N., Hadfield, S., Koller, O., and Bowden, R. (2017, January 22\u201329). SubUNets: End-to-End Hand Shape and Continuous Sign Language Recognition. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.332"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1016\/j.neucom.2017.02.101","article-title":"3D human gesture capturing and recognition by the IMMU-based data glove","volume":"277","author":"Fang","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"6310","DOI":"10.1109\/JSEN.2016.2583542","article-title":"Inertial Motion Sensing Glove for Sign Language Gesture Acquisition and Recognition","volume":"16","author":"Galka","year":"2016","journal-title":"IEEE Sens. J."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Luo, X., Wu, X., Chen, L., Zhao, Y., Zhang, L., Li, G., and Hou, W. (2019). Synergistic Myoelectrical Activities of Forearm Muscles Improving Robust Recognition of Multi-Fingered Gestures. Sensors, 19.","DOI":"10.3390\/s19030610"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3554","DOI":"10.3390\/s18103554","article-title":"American Sign Language Recognition Using Leap Motion Controller with Machine Learning Approach","volume":"18","year":"2018","journal-title":"Sensors"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Su, R., Chen, X., Cao, S., and Zhang, X. (2016). Random Forest-Based Recognition of Isolated Sign Language Subwords Using Data from Accelerometers and Surface Electromyographic Sensors. Sensors, 16.","DOI":"10.3390\/s16010100"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Madushanka, A., Senevirathne, R., Wijesekara, L., and Arunatilake, S. (2016, January 1\u20133). Framework for Sinhala Sign Language recognition and translation using a wearable armband. Proceedings of the 2016 Sixteenth International Conference on Advances in ICT for Emerging Regions (ICTer), Negombo, Sri Lanka.","DOI":"10.1109\/ICTER.2016.7829898"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"208","DOI":"10.1016\/j.eswa.2017.11.049","article-title":"Robust EMG pattern recognition in the presence of confounding factors: Features, classifiers and adaptive learning","volume":"96","author":"Gu","year":"2018","journal-title":"Expert Syst. Appl."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"12431","DOI":"10.3390\/s130912431","article-title":"Surface Electromyography Signal Processing and Classification Techniques","volume":"13","author":"Chowdhury","year":"2013","journal-title":"Sensors"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1251\/bpo115","article-title":"Techniques of EMG signal analysis: Detection, processing, classification and applications","volume":"8","author":"Reaz","year":"2006","journal-title":"Biol. Proced. Online"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"282","DOI":"10.1109\/TMECH.2007.897262","article-title":"A Supervised Feature-Projection-Based Real-Time EMG Pattern Recognition for Multifunction Myoelectric Hand Control","volume":"12","author":"Chu","year":"2007","journal-title":"IEEE\/ASME Trans. Mechatron."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Baddar, W., Kim, D., and Ro, Y. (2017, January 4\u20136). Learning Features Robust to Image Variations with Siamese Networks for Facial Expression Recognition. Proceedings of the International Conference on Multimedia Modeling, Reykjavik, Iceland.","DOI":"10.1007\/978-3-319-51811-4_16"},{"key":"ref_26","first-page":"1106","article-title":"ImageNet Classification with Deep Convolutional Neural Networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27\u201330). Rethinking the Inception Architecture for Computer Vision. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.308"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., Maaten, L., and Weinberger, K. (2017, January 21\u201326). Densely Connected Convolutional Networks. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_30","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1904","DOI":"10.1109\/TPAMI.2015.2389824","article-title":"Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition","volume":"37","author":"He","year":"2014","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Kar, A., Rai, N., Sikka, K., and Sharma, G. (2017, January 21\u201326). AdaScan: Adaptive Scan Pooling in Deep Convolutional Neural Networks for Human Action Recognition in Videos. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.604"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"He, A., Luo, C., Tian, X., and Zeng, W. (2018, January 18\u201323). A twofold siamese network for real-time object tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00508"},{"key":"ref_34","unstructured":"Chopra, S., Hadsell, R., and Lecun, Y. (2005, January 20\u201325). Learning a similarity metric discriminatively, with application to face verification. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201905), San Diego, CA, USA."},{"key":"ref_35","unstructured":"Xu, B., Wang, N., Chen, T., and Li, M. (2015). Empirical Evaluation of Rectified Activations in Convolutional Network. arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Zhang, X., Zou, Y., and Shi, W. (2017, January 23\u201325). Dilated convolution neural network with LeakyReLU for environmental sound classification. Proceedings of the International Conference on Digital Signal Processing (DSP), London, UK.","DOI":"10.1109\/ICDSP.2017.8096153"},{"key":"ref_37","unstructured":"Ioffe, S., and Szegedy, C. (2015, January 6\u201311). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. Proceedings of the 32nd International Conference on International Conference on Machine Learning (ICML), Lille, France."},{"key":"ref_38","unstructured":"Hadsell, R., Chopra, S., and Lecun, Y. (2006, January 17\u201322). Dimensionality Reduction by Learning an Invariant Mapping. Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR\u201906), New York, NY, USA."},{"key":"ref_39","first-page":"249","article-title":"Understanding the difficulty of training deep feedforward neural networks","volume":"9","author":"Glorot","year":"2010","journal-title":"J. Mach. Learn. Res."},{"key":"ref_40","unstructured":"Keskar, N., Mudigere, D., Nocedal, J., Mikhail, S., and Ping, T. (2016). On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1285","DOI":"10.1109\/JBHI.2016.2598302","article-title":"A Wearable System for Recognizing American Sign Language in Real-Time Using IMU and Surface EMG Sensors","volume":"20","author":"Wu","year":"2016","journal-title":"IEEE J. Biomed. Health Inform."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/11\/2495\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:55:06Z","timestamp":1760187306000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/11\/2495"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,5,31]]},"references-count":41,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2019,6]]}},"alternative-id":["s19112495"],"URL":"https:\/\/doi.org\/10.3390\/s19112495","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,5,31]]}}}