{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,21]],"date-time":"2026-03-21T06:27:20Z","timestamp":1774074440832,"version":"3.50.1"},"reference-count":42,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2012,3,1]],"date-time":"2012-03-01T00:00:00Z","timestamp":1330560000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000145","name":"Division of Information and Intelligent Systems","doi-asserted-by":"publisher","award":["IIS-1018055"],"award-info":[{"award-number":["IIS-1018055"]}],"id":[{"id":"10.13039\/100000145","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000006","name":"Office of Naval Research","doi-asserted-by":"publisher","award":["N000140910625"],"award-info":[{"award-number":["N000140910625"]}],"id":[{"id":"10.13039\/100000006","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Interact. Intell. Syst."],"published-print":{"date-parts":[[2012,3]]},"abstract":"<jats:p>Intelligent gesture recognition systems open a new era of natural human-computer interaction: Gesturing is instinctive and a skill we all have, so it requires little or no thought, leaving the focus on the task itself, as it should be, not on the interaction modality. We present a new approach to gesture recognition that attends to both body and hands, and interprets gestures continuously from an unsegmented and unbounded input stream. This article describes the whole procedure of continuous body and hand gesture recognition, from the signal acquisition to processing, to the interpretation of the processed signals.<\/jats:p>\n          <jats:p>Our system takes a vision-based approach, tracking body and hands using a single stereo camera. Body postures are reconstructed in 3D space using a generative model-based approach with a particle filter, combining both static and dynamic attributes of motion as the input feature to make tracking robust to self-occlusion. The reconstructed body postures guide searching for hands. Hand shapes are classified into one of several canonical hand shapes using an appearance-based approach with a multiclass support vector machine. Finally, the extracted body and hand features are combined and used as the input feature for gesture recognition. We consider our task as an online sequence labeling and segmentation problem. A latent-dynamic conditional random field is used with a temporal sliding window to perform the task continuously. We augment this with a novel technique called multilayered filtering, which performs filtering both on the input layer and the prediction layer. Filtering on the input layer allows capturing long-range temporal dependencies and reducing input signal noise; filtering on the prediction layer allows taking weighted votes of multiple overlapping prediction results as well as reducing estimation noise.<\/jats:p>\n          <jats:p>We tested our system in a scenario of real-world gestural interaction using the NATOPS dataset, an official vocabulary of aircraft handling gestures. Our experimental results show that: (1) the use of both static and dynamic attributes of motion in body tracking allows statistically significant improvement of the recognition performance over using static attributes of motion alone; and (2) the multilayered filtering statistically significantly improves recognition performance over the nonfiltering method. We also show that, on a set of twenty-four NATOPS gestures, our system achieves a recognition accuracy of 75.37%.<\/jats:p>","DOI":"10.1145\/2133366.2133371","type":"journal-article","created":{"date-parts":[[2012,6,1]],"date-time":"2012-06-01T15:51:28Z","timestamp":1338565888000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":94,"title":["Continuous body and hand gesture recognition for natural human-computer interaction"],"prefix":"10.1145","volume":"2","author":[{"given":"Yale","family":"Song","sequence":"first","affiliation":[{"name":"Massachusetts Institute of Technology, Cambridge, MA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"David","family":"Demirdjian","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology, Cambridge, MA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Randall","family":"Davis","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology, Cambridge, MA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2012,3,20]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.1981.1673799"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.910878"},{"key":"e_1_2_1_3_1","unstructured":"Bradski G. and Kaehler A. 2008. Learning OpenCV: Computer Vision with the OpenCV Library. O'Reilly Cambridge MA.  Bradski G. and Kaehler A. 2008. Learning OpenCV: Computer Vision with the OpenCV Library. O'Reilly Cambridge MA."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.5555\/850924.851583"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1961189.1961199"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2005.177"},{"key":"e_1_2_1_7_1","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1115\/1.4011045","article-title":"A kinematic notation for lower-pair mechanisms based on matrices","volume":"23","author":"Denavit J.","year":"1955","unstructured":"Denavit , J. and Hartenberg , R. S. 1955 . A kinematic notation for lower-pair mechanisms based on matrices . ASME J. Appl. Mechan. 23 , 215 -- 221 . Denavit, J. and Hartenberg, R. S. 1955. A kinematic notation for lower-pair mechanisms based on matrices. ASME J. Appl. Mechan. 23, 215--221.","journal-title":"ASME J. Appl. Mechan."},{"key":"e_1_2_1_8_1","volume-title":"Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2126--2133","author":"Deutscher J.","unstructured":"Deutscher , J. , Blake , A. , and Reid , I. D . 2000. Articulated body motion capture by annealed particle filtering . In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2126--2133 . Deutscher, J., Blake, A., and Reid, I. D. 2000. Articulated body motion capture by annealed particle filtering. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2126--2133."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/0021-9290(80)90058-5"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cviu.2006.10.012"},{"key":"e_1_2_1_11_1","doi-asserted-by":"crossref","unstructured":"Feng X. Yang J. and Abdel-Malek K. 2008. Survey of biomechanical models for the human shoulder complex. Tech. rep. SAE International.  Feng X. Yang J. and Abdel-Malek K. 2008. Survey of biomechanical models for the human shoulder complex. Tech. rep. SAE International.","DOI":"10.4271\/2008-01-1871"},{"key":"e_1_2_1_12_1","volume-title":"Proceedings of SPIE Machine Vision Applications in Industrial Inspection XII.","author":"Fofi D.","unstructured":"Fofi , D. , Sliwa , T. , and Voisin , Y . 2004. A comparative survey on invisible structured light . In Proceedings of SPIE Machine Vision Applications in Industrial Inspection XII. Fofi, D., Sliwa, T., and Voisin, Y. 2004. A comparative survey on invisible structured light. In Proceedings of SPIE Machine Vision Applications in Industrial Inspection XII."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/38.674971"},{"key":"e_1_2_1_14_1","volume-title":"Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop.","author":"Gokturk S.","unstructured":"Gokturk , S. , Yalcin , H. , and Bamji , C . 2004. A time-of-flight depth sensor\u2014 System description, issues and solutions . In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop. Gokturk, S., Yalcin, H., and Bamji, C. 2004. A time-of-flight depth sensor\u2014 System description, issues and solutions. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop."},{"key":"e_1_2_1_15_1","volume-title":"Proceedings of the 9th European Conference on Speech Communication and Technology. 1117--1120","author":"Gunawardana A.","unstructured":"Gunawardana , A. , Mahajan , M. , Acero , A. , and Platt , J. C . 2005. Hidden conditional random fields for phone classification . In Proceedings of the 9th European Conference on Speech Communication and Technology. 1117--1120 . Gunawardana, A., Mahajan, M., Acero, A., and Platt, J. C. 2005. Hidden conditional random fields for phone classification. In Proceedings of the 9th European Conference on Speech Communication and Technology. 1117--1120."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/PROC.1978.10837"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/72.991427"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.5555\/2041666.2041676"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008078328650"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.rti.2004.12.004"},{"key":"e_1_2_1_21_1","volume-title":"Neurocomputing: Algorithms, Architectures and Applications","author":"Knerr S.","year":"1990","unstructured":"Knerr , S. , Personnaz , L. , and Dreyfus , G . 1990 . Single-Layer learning revisited: A stepwise procedure for building and training a neural network. In Neurocomputing: Algorithms, Architectures and Applications , J. Fogelman, Ed., Springer-Verlag . Knerr, S., Personnaz, L., and Dreyfus, G. 1990. Single-Layer learning revisited: A stepwise procedure for building and training a neural network. In Neurocomputing: Algorithms, Architectures and Applications, J. Fogelman, Ed., Springer-Verlag."},{"key":"e_1_2_1_22_1","volume-title":"Proceedings of the 18th International Conference on Machine Learning. Morgan Kaufmann, 282--289","author":"Lafferty J. D.","unstructured":"Lafferty , J. D. , McCallum , A. , and Pereira , F. C. N. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data . In Proceedings of the 18th International Conference on Machine Learning. Morgan Kaufmann, 282--289 . Lafferty, J. D., McCallum, A., and Pereira, F. C. N. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning. Morgan Kaufmann, 282--289."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2006.110"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCC.2007.893280"},{"key":"e_1_2_1_25_1","volume-title":"Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition.","author":"Morency L.-P.","unstructured":"Morency , L.-P. , Quattoni , A. , and Darrell , T . 2007. Latent-Dynamic discriminative models for continuous gesture recognition . In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Morency, L.-P., Quattoni, A., and Darrell, T. 2007. Latent-Dynamic discriminative models for continuous gesture recognition. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2006.149"},{"key":"e_1_2_1_28_1","volume-title":"Man-Systems Integration Standards","unstructured":"NASA. 1995. Man-Systems Integration Standards : Vol. 1 . Section 3. Anthropometry and Biomechanics . http:\/\/msis.jsc.hasa.gov\/sections\/section03.htm. NASA. 1995. Man-Systems Integration Standards: Vol. 1. Section 3. Anthropometry and Biomechanics. http:\/\/msis.jsc.hasa.gov\/sections\/section03.htm."},{"key":"e_1_2_1_29_1","doi-asserted-by":"crossref","unstructured":"Nocedal J. and Wright S. J. 1999. Numerical Optimization. Springer-Verlag.  Nocedal J. and Wright S. J. 1999. Numerical Optimization. Springer-Verlag.","DOI":"10.1007\/b98874"},{"key":"e_1_2_1_30_1","doi-asserted-by":"crossref","unstructured":"Pearl J. 1988. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann.   Pearl J. 1988. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann.","DOI":"10.1016\/B978-0-08-051489-5.50008-4"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cviu.2006.10.016"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2007.1124"},{"key":"e_1_2_1_33_1","volume-title":"Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. 700--706","author":"Schwarz L. A.","unstructured":"Schwarz , L. A. , Mkhitaryan , A. , Mateus , D. , and Navab , N . 2011. Estimating human 3d pose from time-of-flight images based on geodesic distances and optical flow . In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. 700--706 . Schwarz, L. A., Mkhitaryan, A., Mateus, D., and Navab, N. 2011. Estimating human 3d pose from time-of-flight images based on geodesic distances and optical flow. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. 700--706."},{"key":"e_1_2_1_34_1","volume-title":"Proceedings of the IEEE International Conference on Computer Vision. 750--759","author":"Shakhnarovich G.","unstructured":"Shakhnarovich , G. , Viola , P. A. , and Darrell , T . 2003. Fast pose estimation with parameter-sensitive hashing . In Proceedings of the IEEE International Conference on Computer Vision. 750--759 . Shakhnarovich, G., Viola, P. A., and Darrell, T. 2003. Fast pose estimation with parameter-sensitive hashing. In Proceedings of the IEEE International Conference on Computer Vision. 750--759."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995316"},{"key":"e_1_2_1_36_1","volume-title":"Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 69--76","author":"Sminchisescu C.","unstructured":"Sminchisescu , C. and Triggs , B . 2003. Kinematic jump processes for monocular 3d human tracking . In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 69--76 . Sminchisescu, C. and Triggs, B. 2003. Kinematic jump processes for monocular 3d human tracking. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 69--76."},{"key":"e_1_2_1_37_1","volume-title":"Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. 388--393","author":"Song Y.","unstructured":"Song , Y. , Demirdjian , D. , and Davis , R . 2011a. Multi-Signal gesture recognition using temporal smoothing hidden conditional random fields . In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. 388--393 . Song, Y., Demirdjian, D., and Davis, R. 2011a. Multi-Signal gesture recognition using temporal smoothing hidden conditional random fields. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. 388--393."},{"key":"e_1_2_1_38_1","volume-title":"Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. 500--506","author":"Song Y.","unstructured":"Song , Y. , Demirdjian , D. , and Davis , R . 2011b. Tracking body and hands for gesture recognition: Natops aircraft handling signals database . In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. 500--506 . Song, Y., Demirdjian, D., and Davis, R. 2011b. Tracking body and hands for gesture recognition: Natops aircraft handling signals database. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. 500--506."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015330.1015422"},{"key":"e_1_2_1_40_1","volume-title":"The Nature of Statistical Learning Theory","author":"Vapnik V. N.","unstructured":"Vapnik , V. N. 1995. The Nature of Statistical Learning Theory . Springer , New York . Vapnik, V. N. 1995. The Nature of Statistical Learning Theory. Springer, New York."},{"key":"e_1_2_1_41_1","volume-title":"Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 872--879","author":"Wang Y.","unstructured":"Wang , Y. and Mori , G . 2009. Max-Margin hidden conditional random fields for human action recognition . In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 872--879 . Wang, Y. and Mori, G. 2009. Max-Margin hidden conditional random fields for human action recognition. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 872--879."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cviu.2010.10.002"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/1891903.1891924"}],"container-title":["ACM Transactions on Interactive Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2133366.2133371","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2133366.2133371","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T10:06:05Z","timestamp":1750241165000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2133366.2133371"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,3]]},"references-count":42,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2012,3]]}},"alternative-id":["10.1145\/2133366.2133371"],"URL":"https:\/\/doi.org\/10.1145\/2133366.2133371","relation":{},"ISSN":["2160-6455","2160-6463"],"issn-type":[{"value":"2160-6455","type":"print"},{"value":"2160-6463","type":"electronic"}],"subject":[],"published":{"date-parts":[[2012,3]]},"assertion":[{"value":"2010-12-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2011-12-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2012-03-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}