{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,27]],"date-time":"2025-10-27T16:06:59Z","timestamp":1761581219176},"reference-count":46,"publisher":"MIT Press - Journals","issue":"2","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Presence: Teleoperators and Virtual Environments"],"published-print":{"date-parts":[[2014,8,1]]},"abstract":"<jats:p> In this paper, a human\u2013robot interaction system based on a novel combination of sensors is proposed. It allows one person to interact with a humanoid social robot using natural body language. The robot understands the meaning of human upper body gestures and expresses itself by using a combination of body movements, facial expressions, and verbal language. A set of 12 upper body gestures is involved for communication. This set also includes gestures with human\u2013object interactions. The gestures are characterized by head, arm, and hand posture information. The wearable Immersion CyberGlove II is employed to capture the hand posture. This information is combined with the head and arm posture captured from Microsoft Kinect. This is a new sensor solution for human-gesture capture. Based on the posture data from the CyberGlove II and Kinect, an effective and real-time human gesture recognition method is proposed. The gesture understanding approach based on an innovative combination of sensors is the main contribution of this paper. To verify the effectiveness of the proposed gesture recognition method, a human body gesture data set is built. The experimental results demonstrate that our approach can recognize the upper body gestures with high accuracy in real time. In addition, for robot motion generation and control, a novel online motion planning method is proposed. In order to generate appropriate dynamic motion, a quadratic programming (QP)-based dual-arms kinematic motion generation scheme is proposed, and a simplified recurrent neural network is employed to solve the QP problem. The integration of a handshake within the HRI system illustrates the effectiveness of the proposed online generation method. <\/jats:p>","DOI":"10.1162\/pres_a_00176","type":"journal-article","created":{"date-parts":[[2014,8,4]],"date-time":"2014-08-04T18:03:14Z","timestamp":1407175394000},"page":"133-154","source":"Crossref","is-referenced-by-count":58,"title":["Human\u2013Robot Interaction by Understanding Upper Body Gestures"],"prefix":"10.1162","volume":"23","author":[{"given":"Yang","family":"Xiao","sequence":"first","affiliation":[{"name":"Institute for Media Innovation Nanyang Technological University Singapore"}]},{"given":"Zhijun","family":"Zhang","sequence":"additional","affiliation":[{"name":"Institute for Media Innovation Nanyang Technological University Singapore"}]},{"given":"Aryel","family":"Beck","sequence":"additional","affiliation":[{"name":"Institute for Media Innovation Nanyang Technological University Singapore"}]},{"given":"Junsong","family":"Yuan","sequence":"additional","affiliation":[{"name":"School of Electrical and Electronic Engineering Nanyang Technological University Singapore"}]},{"given":"Daniel","family":"Thalmann","sequence":"additional","affiliation":[{"name":"Institute for Media Innovation Nanyang Technological University Singapore"}]}],"member":"281","reference":[{"key":"p_1","first-page":"9","volume":"2007","author":"Adamo-Villani N.","year":"2007","journal-title":"Proceedings of the First International Conference on Immersive Telecommunications, ICIT"},{"key":"p_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12369-013-0193-z"},{"key":"p_3","first-page":"1845","volume":"2013","author":"Beck A.","year":"2013","journal-title":"COG SCI"},{"key":"p_4","doi-asserted-by":"publisher","DOI":"10.1145\/2133366.2133368"},{"key":"p_5","doi-asserted-by":"publisher","DOI":"10.1109\/34.993558"},{"key":"p_6","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCC.2011.2161077"},{"key":"p_8","doi-asserted-by":"publisher","DOI":"10.1109\/TIE.2011.2106092"},{"key":"p_9","doi-asserted-by":"publisher","DOI":"10.1109\/3468.952719"},{"key":"p_11","doi-asserted-by":"publisher","DOI":"10.1109\/70.370511"},{"key":"p_12","doi-asserted-by":"publisher","DOI":"10.1109\/70.285587"},{"key":"p_14","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.1967.1053964"},{"key":"p_15","doi-asserted-by":"publisher","DOI":"10.1098\/rstb.2006.2004"},{"key":"p_16","first-page":"891","volume":"2009","author":"Faber F.","year":"2009","journal-title":"Proceedings of the IEEE Symposium on Robot and Human Interactive Communication, RO-MAN"},{"key":"p_17","doi-asserted-by":"publisher","DOI":"10.1111\/j.1469-1809.1936.tb02137.x"},{"key":"p_18","doi-asserted-by":"publisher","DOI":"10.1016\/S0921-8890(02)00372-X"},{"key":"p_19","doi-asserted-by":"publisher","DOI":"10.1561\/1100000005"},{"key":"p_20","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCC.2012.2183868"},{"key":"p_23","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2011.2142450"},{"key":"p_25","doi-asserted-by":"publisher","DOI":"10.1007\/s10055-011-0195-9"},{"key":"p_26","doi-asserted-by":"publisher","DOI":"10.1163\/15685530260425729"},{"key":"p_27","first-page":"169","volume":"2006","author":"Martins A.","year":"2006","journal-title":"LARS"},{"key":"p_29","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2004.07.008"},{"key":"p_30","first-page":"137","volume":"2006","author":"M\u00fcller M.","year":"2006","journal-title":"SCA"},{"key":"p_31","doi-asserted-by":"publisher","DOI":"10.1016\/j.imavis.2005.12.020"},{"key":"p_33","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2001.1183338"},{"key":"p_34","first-page":"4013","volume":"2009","author":"Pierris G.","year":"2009","journal-title":"ICRA"},{"key":"p_35","first-page":"1297","volume":"2011","author":"Shotton J.","year":"2011","journal-title":"CVPR"},{"key":"p_36","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-7687.2007.00565.x"},{"key":"p_37","first-page":"25","author":"Spiliotopoulos D.","year":"2001","journal-title":"Proceedings of the European Workshop on Service and Humanoid Robots"},{"key":"p_38","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2007.907484"},{"key":"p_40","doi-asserted-by":"publisher","DOI":"10.1163\/156855308X305263"},{"key":"p_41","first-page":"1","volume":"2012","author":"Takahashi Y.","year":"2012","journal-title":"Proceedings of the IEEE Conference on Fuzzy Systems, FUZZ-IEEE"},{"key":"p_42","first-page":"107","volume":"2012","author":"Teleb H.","year":"2012","journal-title":"Proceedings of the International Conference on Systems and Informatics, ICSAI"},{"key":"p_43","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008918401478"},{"key":"p_44","first-page":"516","volume":"2009","author":"Wang J.","year":"2009","journal-title":"Proceedings of the International Conference on Information and Automation, ICIA"},{"key":"p_45","first-page":"1290","volume":"2012","author":"Wang J.","year":"2012","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR"},{"key":"p_46","first-page":"207","volume":"10","author":"Weinberger K. Q.","year":"2009","journal-title":"The Journal of Machine Learning Research"},{"key":"p_47","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2013.12.017"},{"key":"p_48","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2013.2295756"},{"key":"p_49","doi-asserted-by":"publisher","DOI":"10.1145\/2503713.2503727"},{"key":"p_50","first-page":"19","volume":"2008","author":"Zhang Y.","year":"2008","journal-title":"Proceedings of the IEEE Joint Conference on Neural Networks, IJCNN"},{"key":"p_51","doi-asserted-by":"publisher","DOI":"10.1016\/j.rcim.2012.09.004"},{"key":"p_52","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCB.2012.2189003"},{"issue":"1","key":"p_53","first-page":"25","volume":"7","author":"Zhang Z.","year":"2013","journal-title":"IET"},{"key":"p_54","doi-asserted-by":"publisher","DOI":"10.3724\/SP.J.1004.2013.00088"},{"key":"p_55","doi-asserted-by":"publisher","DOI":"10.1109\/TMECH.2011.2181977"}],"container-title":["Presence: Teleoperators and Virtual Environments"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/PRES_a_00176","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:29:29Z","timestamp":1615584569000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/pvar\/article\/23\/2\/133-154\/59061"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,8,1]]},"references-count":46,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2014,8,1]]}},"alternative-id":["10.1162\/PRES_a_00176"],"URL":"https:\/\/doi.org\/10.1162\/pres_a_00176","relation":{},"ISSN":["1054-7460"],"issn-type":[{"value":"1054-7460","type":"print"}],"subject":[],"published":{"date-parts":[[2014,8,1]]}}}