{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:34:41Z","timestamp":1760243681605,"version":"build-2065373602"},"reference-count":31,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2013,12,12]],"date-time":"2013-12-12T00:00:00Z","timestamp":1386806400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/3.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This study proposes a dynamic hand gesture detection technology to effectively detect dynamic hand gesture areas, and a hand gesture recognition technology to improve the dynamic hand gesture recognition rate. Meanwhile, the corresponding relationship between state sequences in hand gesture and speech models is considered by integrating speech recognition technology with a multimodal model, thus improving the accuracy of human behavior recognition. The experimental results proved that the proposed method can effectively improve human behavior recognition accuracy and the feasibility of system applications. Experimental results verified that the multimodal gesture-speech model provided superior accuracy when compared to the single modal versions.<\/jats:p>","DOI":"10.3390\/s131217098","type":"journal-article","created":{"date-parts":[[2013,12,12]],"date-time":"2013-12-12T13:35:11Z","timestamp":1386855311000},"page":"17098-17129","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Combined Hand Gesture \u2014 Speech Model for Human  Action Recognition"],"prefix":"10.3390","volume":"13","author":[{"given":"Sheng-Tzong","family":"Cheng","sequence":"first","affiliation":[{"name":"Department of Computer Science and Information Engineering, National Cheng Kung University,  No.1, University Road, Tainan City 701, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chih-Wei","family":"Hsu","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Information Engineering, National Cheng Kung University,  No.1, University Road, Tainan City 701, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jian-Pan","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Information Engineering, National Cheng Kung University,  No.1, University Road, Tainan City 701, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2013,12,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Calinon, S., and Billard, A. (2007, January 9\u201311). Incremental Learning of Gestures by Imitation in a Humanoid Robot. Arlington, VA, USA.","DOI":"10.1145\/1228716.1228751"},{"key":"ref_2","unstructured":"Calinon, S., and Billard, A. (2008, January 19\u201323). A Framework Integrating Statistical and Social Cues to Teach a Humanoid Robot New Skills. Pasadena, CA, USA."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1505","DOI":"10.1109\/TIM.2011.2108075","article-title":"Human-computer interaction for smart environment applications using fuzzy hand posture and gesture models","volume":"60","author":"Tusor","year":"2011","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_4","unstructured":"Manresa, C., Varona, J., Mas, R., and Perales, F.J. (2000). Real-time hand tracking and gesture recognition for human-computer interaction. Electron. Lett. Comput. Vis. Image Anal., 1\u20137."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"3592","DOI":"10.1109\/TIM.2011.2161140","article-title":"Real-time hand gesture detection and recognition using bag-of-features and support vector machine techniques","volume":"60","author":"Dardas","year":"2011","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1110","DOI":"10.1109\/TMM.2013.2246148","article-title":"Robust part-based hand gesture recognition using kinect sensor","volume":"15","author":"Ren","year":"2013","journal-title":"IEEE Trans. Multimed."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1562","DOI":"10.1109\/TIM.2008.922070","article-title":"Hand gesture recognition using haar-like features and a stochastic context-free grammar","volume":"57","author":"Chen","year":"2008","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"871","DOI":"10.1109\/TSMCB.2012.2217324","article-title":"Most probable longest common subsequence for recognition of gesture character input","volume":"43","author":"Frolova","year":"2013","journal-title":"IEEE Trans. Cybern."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1064","DOI":"10.1109\/TSMCA.2011.2116004","article-title":"A framework for hand gesture recognition based on accelerometer and EMG sensors","volume":"41","author":"Zhang","year":"2011","journal-title":"IEEE Trans. Syst. Man Cybern. Part A Syst. Hum."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"932","DOI":"10.1109\/TSMCA.2005.851332","article-title":"Cluster labeling and parameter estimation for the automated setup of a hand-gesture recognition system","volume":"35","author":"Stern","year":"2005","journal-title":"IEEE Trans. Syst. Man Cybern. Part A Syst. Hum."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1166","DOI":"10.1109\/JSEN.2011.2166953","article-title":"MEMS accelerometer based nonspecific-user hand gesture recognition","volume":"12","author":"Xu","year":"2012","journal-title":"IEEE Sens. J."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Tang, H.-K., and Feng, Z.-Q. (2008, January 20\u201322). Hand's Skin Detection Based on Ellipse Clustering. Shanghai, China.","DOI":"10.1109\/ISCSCT.2008.53"},{"key":"ref_13","unstructured":"Zhang, M.-J., Wang, W.-Q., Zheng, Q.-F., and Gao, W. (2004, January 18\u201320). Skin-Color Detection Based on Adaptive Thresholds. Washington, DC, USA."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Cai, X., Jiang, L., Hao, X.-W., and Meng, X.-X. (2008, January 18\u201320). A New Region Gaussian Background Model for Video Surveillance. Jinan, China.","DOI":"10.1109\/ICNC.2008.626"},{"key":"ref_15","unstructured":"Wang, S.B. (2008). Video Completion Based on Effective Spatial and Temporal Inpainting Techniques. [M.Sc. Thesis, Department of Information Engineering]."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Yoon, H.-S., and Chi, S.-Y. (2006, January 18\u201321). Visual Processing of Rock, Scissors, Paper Game for Human Robot Interaction. Busan, Korea.","DOI":"10.1109\/SICE.2006.315701"},{"key":"ref_17","unstructured":"Han, J., Award, G.M., Sutherland, A., and Wu, H. (2006, January 10\u201312). Automatic Skin Segmentation for Gesture Recognition Combining Region and Support Vector Machine Active Learning. Southampton, UK."},{"key":"ref_18","unstructured":"Hayakawa, H., and Shibata, T. (2008, January 18\u201321). Spatiotemporal Projection of Motion Field Sequence for Generating Feature Vectors in Gesture Perception. Seattle, WA, USA."},{"key":"ref_19","unstructured":"Lin, W., Sun, M.-T., Poovandran, R., and Zhang, Z. (2008, January 18\u201321). Human Activity Recognition for Video Surveillance. Seattle, WA, USA."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"7714","DOI":"10.3390\/s130607714","article-title":"Geometric feature-based facial expression recognition in image sequences using multi-class adaboost and support vector machines","volume":"13","author":"Ghimire","year":"2013","journal-title":"Sensors"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1130","DOI":"10.3390\/s120201130","article-title":"Hand motion classification using a multi-channel surface electromyography sensor","volume":"12","author":"Tang","year":"2012","journal-title":"Sensors"},{"key":"ref_22","unstructured":"Henry, P., Krainin, M., Herbst, E., Ren, X., and Fox, D. (2010, January 18\u201320). RGB-D Mapping: Using Depth Cameras for Dense 3D Modeling of Indoor Environments. Delhi, India."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"647","DOI":"10.1177\/0278364911434148","article-title":"RGB-D Mapping: Using kinect-style depth cameras for dense 3D modeling of indoor environments","volume":"31","author":"Henry","year":"2012","journal-title":"Int. J. Robot. Res."},{"key":"ref_24","unstructured":"Yang, X., Yuan, J., and Thalmann, D. (2013, January 6\u20139). Human-Virtual Human Interaction by Upper Body Gesture Understanding. Singapore."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1016\/j.imavis.2011.11.003","article-title":"Dynamic hand gesture recognition: An exemplar-based approach from motion divergence fields","volume":"30","author":"Shen","year":"2012","journal-title":"Image Vis. Comput."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"745","DOI":"10.1016\/S0262-8856(03)00070-2","article-title":"Hand gesture recognition using a real-time tracking method and hidden Markov models","volume":"21","author":"Chen","year":"2003","journal-title":"Image Vis. Comput."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"369","DOI":"10.1109\/76.709403","article-title":"A Novel unrestricted center-biased diamond search algorithm for block motion estimation","volume":"8","author":"Tham","year":"1998","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_28","unstructured":"Chai, D., and Bouzerdoum, A. (2000, January 24\u201327). A Bayesian Approach to Skin Color Classification in YCbCr Color Space. Renaissance-New World Hotel, Kuala Lumpur, Malaysia."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Liu, C.-S., Lin, J.-C., Yang, N.-C., and Kuo, C.-M. (2007, January 26\u201328). Motion Vector Re-estimation for Trans-coding Using Kalman Filter. Kaohsiung, Taiwan.","DOI":"10.1109\/IIH-MSP.2007.223"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1109\/TCSVT.2005.857287","article-title":"Kalman filtering based rate-constrained motion estimation for very low bit rate video coding","volume":"16","author":"Kuo","year":"2006","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_31","unstructured":"Young, S., Evermann, G., Gales, M., Hain, T., Kershaw, D., Liu, X., Moore, G., Odell, J., Ollason, D., and Povey, D. (2009). The HTK Book (for HTK version 3.4); Engineering Department, Cambridge University."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/13\/12\/17098\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T21:51:18Z","timestamp":1760219478000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/13\/12\/17098"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2013,12,12]]},"references-count":31,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2013,12]]}},"alternative-id":["s131217098"],"URL":"https:\/\/doi.org\/10.3390\/s131217098","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2013,12,12]]}}}