{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,24]],"date-time":"2026-03-24T18:38:34Z","timestamp":1774377514330,"version":"3.50.1"},"reference-count":45,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2013,9,17]],"date-time":"2013-09-17T00:00:00Z","timestamp":1379376000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/3.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The main activity of social robots is to interact with people. In order to do that, the robot must be able to understand what the user is saying or doing. Typically, this capability consists of pre-programmed behaviors or is acquired through controlled learning processes, which are executed before the social interaction begins. This paper presents a software architecture that enables a robot to learn poses in a similar way as people do. That is, hearing its teacher\u2019s explanations and acquiring new knowledge in real time. The architecture leans on two main components: an RGB-D (Red-, Green-, Blue- Depth) -based visual system, which gathers the user examples, and an Automatic Speech Recognition (ASR) system, which processes the speech describing those examples. The robot is able to naturally learn the poses the teacher is showing to it by maintaining a natural interaction with the teacher. We evaluate our system with 24 users who teach the robot a predetermined set of poses. The experimental results show that, with a few training examples, the system reaches high accuracy and robustness. This method shows how to combine data from the visual and auditory systems for the acquisition of new knowledge in a natural manner. Such a natural way of training enables robots to learn from users, even if they are not experts in robotics.<\/jats:p>","DOI":"10.3390\/s130912406","type":"journal-article","created":{"date-parts":[[2013,9,17]],"date-time":"2013-09-17T12:31:17Z","timestamp":1379421077000},"page":"12406-12430","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Teaching Human Poses Interactively to a Social Robot"],"prefix":"10.3390","volume":"13","author":[{"given":"Victor","family":"Gonzalez-Pacheco","sequence":"first","affiliation":[{"name":"Robotics Lab, Systems Engineering and Automation Department, Universidad Carlos III de Madrid, Av. Universidad 30, Legan\u00e9s 28911, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Maria","family":"Malfaz","sequence":"additional","affiliation":[{"name":"Robotics Lab, Systems Engineering and Automation Department, Universidad Carlos III de Madrid, Av. Universidad 30, Legan\u00e9s 28911, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fernando","family":"Fernandez","sequence":"additional","affiliation":[{"name":"Computer Science Department, Universidad Carlos III de Madrid, Av. Universidad 30, Legan\u00e9s 28911, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Miguel","family":"Salichs","sequence":"additional","affiliation":[{"name":"Robotics Lab, Systems Engineering and Automation Department, Universidad Carlos III de Madrid, Av. Universidad 30, Legan\u00e9s 28911, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2013,9,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1109\/TSMCC.2007.893280","article-title":"Gesture Recognition: A Survey","volume":"37","author":"Mitra","year":"2007","journal-title":"IEEE Trans. Syst. Man Cybern. C"},{"key":"ref_2","unstructured":"Jaume-I Cap\u00f3, A., and Varona, J. (2009). Gesture-Based Human-Computer Interaction and Simulation, Springer."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1917","DOI":"10.1109\/JSEN.2010.2101060","article-title":"Lock-in Time-of-Flight (ToF) cameras: A survey","volume":"11","author":"Foix","year":"2011","journal-title":"IEEE Sens. J."},{"key":"ref_4","unstructured":"Scharstein, D., and Szeliski, R. (2003, January 16\u201322). High-Accuracy Stereo Depth Maps Using Structured Light. Madison, WI, USA."},{"key":"ref_5","unstructured":"Freedman, B., Shpunt, A., Machline, M., and Arieli, Y. (2010). Depth Mapping Using Projected Patterns. (US Patent 2010\/0118123 A1)."},{"key":"ref_6","unstructured":"Wikiepdia Contributors Kinect. Available online: http:\/\/en.wikipedia.org\/wiki\/Kinect."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Salichs, M., Barber, R., Khamis, A., Malfaz, M., Gorostiza, J., Pacheco, R., Rivas, R., Corrales, A., Delgado, E., and Garcia, D. (2006, January 7\u20139). Maggie: A Robotic Platform for Human\u2013Robot Social Interaction. Bangkok, Thailand.","DOI":"10.1109\/RAMECH.2006.252754"},{"key":"ref_8","unstructured":"Kolb, A., Barth, E., Koch, R., and Larsen, R. (2009). Time-of-Flight Sensors in Computer Graphics, The Eurographics Association. Eurographics State of the Art Reports."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1995","DOI":"10.1016\/j.patrec.2013.02.006","article-title":"A survey of human motion analysis using depth imagery","volume":"34","author":"Chen","year":"2013","journal-title":"Pattern Recognit. Lett."},{"key":"ref_10","unstructured":"Gokturk, S.B., and Tomasi, C. (July, January 27). 3D Head Tracking Based on Recognition and Interpolation Using a Time-of-Flight Depth Sensor. Washington, DC, USA."},{"key":"ref_11","unstructured":"Liu, X., and Fujimura, K. (2004, January 17\u201319). Hand Gesture Recognition Using Depth Data. Washington, DC, USA."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1007\/978-3-540-71457-6_23","article-title":"Hand Gesture Recognition with a Novel IR Time-of-Flight Range Camera\u2014A Pilot Study","volume":"Volume 4418","author":"Gagalowicz","year":"2007","journal-title":"Computer Vision\/Computer Graphics Collaboration Techniques"},{"key":"ref_13","unstructured":"Lahamy, H., and Litchi, D. (2010, January 15\u201318). Real-Time Hand Gesture Recognition Using Range Cameras. Calgary, Canada."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1875","DOI":"10.1016\/j.imavis.2005.12.020","article-title":"Visual recognition of pointing gestures for human\u2013robot interaction","volume":"25","author":"Nickel","year":"2007","journal-title":"Image Vis. Comput."},{"key":"ref_15","unstructured":"Haubner, N., Schwanecke, U., D\u00f6rner, R., Lehmann, S., and Luderschmidt, J. (2010, January 12). Recognition of Dynamic Hand Gestures with Time-of-Flight Cameras. Wiesbaden, Germany."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Droeschel, D., St\u00fcckler, J., and Behnke, S. (2011, January 6\u20138). Learning to Interpret Pointing Gestures with a Time-of-Flight Camera. Lausanne, Switzerland.","DOI":"10.1145\/1957656.1957822"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1007\/s10055-006-0024-8","article-title":"Evaluation of on-line analytic and numeric inverse kinematics approaches driven by partial vision input","volume":"10","author":"Boulic","year":"2006","journal-title":"Virtual Real."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1362","DOI":"10.1016\/j.cviu.2009.11.005","article-title":"Kinematic self retargeting: A framework for human pose estimation","volume":"114","author":"Zhu","year":"2010","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ramey, A., Gonz\u00e1lez-Pacheco, V., and Salichs, M.A. (2011, January 6\u20138). Integration of a Low-Cost RGB-D Sensor in a Social Robot for Gesture Recognition. Lausanne, Switzerland.","DOI":"10.1145\/1957656.1957745"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1016\/j.imavis.2011.12.001","article-title":"Human skeleton tracking from depth data using geodesic distances and optical flow","volume":"30","author":"Schwarz","year":"2011","journal-title":"Image Vision Comput."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., and Blake, A. (2011, January 21\u201323). Real-Time Human Pose Recognition in Parts From Single Depth Images. Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995316"},{"key":"ref_22","unstructured":"OpenNI Available online: http:\/\/www.openni.org\/."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1016\/S0921-8890(02)00372-X","article-title":"A survey of socially interactive robots","volume":"42","author":"Fong","year":"2003","journal-title":"Robot. Auton. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1561\/1100000005","article-title":"Human\u2013robot interaction: A survey","volume":"1","author":"Goodrich","year":"2007","journal-title":"Found. Trends Hum. Comput. Interact."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1016\/j.robot.2008.10.024","article-title":"A survey of robot learning from demonstration","volume":"57","author":"Argall","year":"2009","journal-title":"Robot. Auton. Syst."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1023\/A:1008850021368","article-title":"Rapid concept learning for mobile robots","volume":"5","author":"Mahadevan","year":"1998","journal-title":"Auton. Robots"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"De Greeff, J., Delaunay, F., and Belpaeme, T. (2009, January 4\u20137). Human\u2013Robot Interaction in Concept Acquisition: A Computational Model. Shangai, China.","DOI":"10.1109\/DEVLRN.2009.5175532"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1109\/3468.952716","article-title":"Learning and interacting in human\u2013robot domains","volume":"31","author":"Nicolescu","year":"2001","journal-title":"IEEE Trans. Syst. Man Cybern. A"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Rybski, P.E., Yoon, K., Stolarz, J., and Veloso, M.M. (2007, January 9\u201311). Interactive Robot Task Training through Dialog and Demonstration. Washington, DC, USA.","DOI":"10.1145\/1228716.1228724"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Goerick, C., Schmudderich, J., Bolder, B., Janssen, H., Gienger, M., Bendig, A., Heckmann, M., Rodemann, T., Brandl, H., and Domont, X. (2009, January 7\u201310). Interactive Online Multimodal Association for Internal Concept Building in Humanoids. Paris, France.","DOI":"10.1109\/ICHR.2009.5379549"},{"key":"ref_31","unstructured":"Heckmann, M., Brandl, H., Schmuedderich, J., Domont, X., Bolder, B., Mikhailova, I., Janssen, H., Gienger, M., Bendig, A., and Rodemann, T. (October, January 27). Teaching a Humanoid Robot: Headset-Free Speech Interaction for Audio-Visual Association Learning. Toyama, Japan."},{"key":"ref_32","unstructured":"Zalevsky, Z., Shpunt, A., Maizels, A., and Garcia, J. (2013). Method and System for Object Reconstruction. (US Patent US20130155195 A1)."},{"key":"ref_33","unstructured":"Barber, R. (2000). Desarrollo de una Arquitectura Para Robots M\u00f3viles Aut\u00f3nomos. Aplicacio\u00f3n a un Sistema de Navegaci\u00f3n Topolo\u00f3gica. [Ph.D. Thesis, Universidad Carlos III de Madrid]."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Rivas, R., Corrales, A., Barber, R., and Salichs, MA. (2007, January 3\u20135). Robot Skill Abstraction for AD Architecture. Toulouse, France.","DOI":"10.3182\/20070903-3-FR-2921.00071"},{"key":"ref_35","unstructured":"Gamma, E., Helm, R., Johnson, R., and Vlissides, J. (1995). Design Patterns: Elements of Reusable Object-Oriented Software, Addison-Wesley Professional."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Quigley, M., Gerkey, B., Conley, K., Faust, J., Foote, T., Leibs, J., Berger, E., Wheeler, R., and Ng, A. (2009, January 12\u201317). ROS: An Open-Source Robot Operating System. Kobe, Japan.","DOI":"10.1109\/MRA.2010.936956"},{"key":"ref_37","unstructured":"Nuance Communications LTD. Loquendo web page. Available online: www.loquendo.com\/."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1080\/01969722.2011.583593","article-title":"Integration of a voice recognition system in a social robot","volume":"42","author":"Salichs","year":"2011","journal-title":"Cybern. Syst."},{"key":"ref_39","unstructured":"Alonso-Martin, F., Ramey, A.A., and Salichs, M.A. (, January May). Maggie: El Robot Traductor. Madrid, Spain."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1145\/1656274.1656278","article-title":"The WEKA data mining software: An update","volume":"11","author":"Hall","year":"2009","journal-title":"ACM SIGKDD Explor. Newsl."},{"key":"ref_41","unstructured":"Witten, I.H., Frank, E., and Hall, M.A. (2011). Data Mining: Practical Machine Learning Tools and Techniques, Elsevier Science & Technology. [3rd ed.]."},{"key":"ref_42","unstructured":"Quinlan, J.R. (1993). C4.5: Programs for Machine Learning, Morgan Kaufmann Publishers."},{"key":"ref_43","unstructured":"John, G.H., and Langley, P. (1995, January 18\u201320). Estimating Continuous Distributions in Bayesian Classifiers. Montreal, QC, Canada."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1188","DOI":"10.1109\/72.870050","article-title":"Improvements to the SMO algorithm for SVM regression","volume":"11","author":"Shevade","year":"2000","journal-title":"IEEE Trans. Neural Netw."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/13\/9\/12406\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T21:49:21Z","timestamp":1760219361000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/13\/9\/12406"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2013,9,17]]},"references-count":45,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2013,9]]}},"alternative-id":["s130912406"],"URL":"https:\/\/doi.org\/10.3390\/s130912406","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2013,9,17]]}}}