{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:34:57Z","timestamp":1760243697020,"version":"build-2065373602"},"reference-count":48,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2012,7,23]],"date-time":"2012-07-23T00:00:00Z","timestamp":1343001600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/3.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper presents a user localization system based on the fusion of visual information and sound source localization, implemented on a social robot called Maggie. One of the main requisites to obtain a natural interaction between human-human and human-robot is an adequate spatial situation between the interlocutors, that is, to be orientated and situated at the right distance during the conversation in order to have a satisfactory communicative process. Our social robot uses a complete multimodal dialog system which manages the user-robot interaction during the communicative process. One of its main components is the presented user localization system. To determine the most suitable allocation of the robot in relation to the user, a proxemic study of the human-robot interaction is required, which is described in this paper. The study has been made with two groups of users: children, aged between 8 and 17, and adults. Finally, at the end of the paper, experimental results with the proposed multimodal dialog system are presented.<\/jats:p>","DOI":"10.3390\/s120709913","type":"journal-article","created":{"date-parts":[[2012,7,23]],"date-time":"2012-07-23T11:11:45Z","timestamp":1343041905000},"page":"9913-9935","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["User Localization During Human-Robot Interaction"],"prefix":"10.3390","volume":"12","author":[{"given":"F.","family":"Alonso-Mart\u00edn","sequence":"first","affiliation":[{"name":"Robotics Lab, Universidad Carlos III de Madrid, Av. de la Universidad 30, 28911 Legan\u00e9s, Madrid, Spain"}]},{"given":"Javi F.","family":"Gorostiza","sequence":"additional","affiliation":[{"name":"Robotics Lab, Universidad Carlos III de Madrid, Av. de la Universidad 30, 28911 Legan\u00e9s, Madrid, Spain"}]},{"given":"Mar\u00eda","family":"Malfaz","sequence":"additional","affiliation":[{"name":"Robotics Lab, Universidad Carlos III de Madrid, Av. de la Universidad 30, 28911 Legan\u00e9s, Madrid, Spain"}]},{"given":"Miguel A.","family":"Salichs","sequence":"additional","affiliation":[{"name":"Robotics Lab, Universidad Carlos III de Madrid, Av. de la Universidad 30, 28911 Legan\u00e9s, Madrid, Spain"}]}],"member":"1968","published-online":{"date-parts":[[2012,7,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"727","DOI":"10.1126\/science.3945806","article-title":"The cyclopean ear: A new sense for the praying mantis","volume":"231","author":"Yager","year":"1986","journal-title":"Science"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1038\/scientificamerican0183-54","article-title":"The hair cells of the inner ear","volume":"248","author":"Hudspeth","year":"1983","journal-title":"Sci. Am."},{"key":"ref_3","first-page":"327","article-title":"The sense of rotation and the anatomy and physiology of the semicircular canals of the internal ear","volume":"8","author":"Brown","year":"1874","journal-title":"J. Anat. Physiol."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Dooling, R., and Popper, A. (2000). Comparative Hearing: Reptiles and Birds, Springer-Verlag.","DOI":"10.1007\/978-1-4612-1182-2"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1156","DOI":"10.1139\/z78-159","article-title":"Detection of substrate vibrations by salamanders: Inner ear sense organ activity","volume":"56","author":"Ross","year":"1978","journal-title":"Can. J. Zool."},{"key":"ref_6","unstructured":"Nakadai, K., Matsuura, D., Okuno, H., and Kitano, H. (October, January 27\u2013). Applying Scattering Theory to Robot Audition System: Robust Sound Source Localization and Extraction. Las Vegas, NV, USA. Volume 2."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Nakadai, K., Okuno, H., and Kitano, H. (2002, January 16\u201320). Real-Time Sound Source Localization and Separation for Robot Audition. Denver, CO, USA.","DOI":"10.21437\/ICSLP.2002-118"},{"key":"ref_8","unstructured":"Valin, J., Michaud, F., Rouat, J., and L\u00e9tourneau, D. (October, January 27\u2013). Robust Sound Source Localization Using a Microphone Array on A Mobile Robot. Las Vegas, NV, USA. Volume 2."},{"key":"ref_9","unstructured":"Murray, J., Erwin, H., and Wermter, S. (2004, January 20). Robotics Sound-Source Localization and Tracking Using Interaural Time Difference and Crosscorrelation. Ulm, Germany."},{"key":"ref_10","unstructured":"Andersson, S., Handzel, A., Shah, V., and Krishnaprasad, P. (1, January 26). Robot Phonotaxis with Dynamic Sound-Source Localization. New Orleans, LA, USA. Volume 5."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1080\/01969722.2011.583593","article-title":"Integration of a voice recognition system in a social robot","volume":"42","author":"Salichs","year":"2011","journal-title":"Cybern. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"12307","DOI":"10.1073\/pnas.211220498","article-title":"Different patterns of human discrimination learning for two interaural cues to sound-source location","volume":"98","author":"Wright","year":"2001","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1109\/JSEN.2002.807772","article-title":"Biomimetic sound-source localization","volume":"2","author":"Handzel","year":"2002","journal-title":"IEEE Sens. J."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Brandstein, M., and Ward, D. (2001). Microphone Arrays: Signal Processing Techniques And Applications, Springer Verlag.","DOI":"10.1007\/978-3-662-04619-7"},{"key":"ref_15","unstructured":"Benesty, J., Chen, J., and Huang, Y. (2008). Microphone Array Signal Processing, Springer Verlag. Volume 1."},{"key":"ref_16","unstructured":"Saxena, A., and Ng, A.Y. (May, January 12\u2013). Learning Sound Location from a Single Microphone. Kobe, Japan."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"597","DOI":"10.1121\/1.398749","article-title":"Infants' monaural localization of sounds: Effects of unilateral ear infection","volume":"86","author":"Morrongiello","year":"1989","journal-title":"J. Acoust. Soc. Am."},{"key":"ref_18","unstructured":"Hu, J.-S., Chan, C.-Y., Wang, C.-K., and Wang, C.-C. (May, January 12\u2013). Simultaneous Localization of Mobile Robot and Multiple Sound Sources Using Microphone Array. Kobe, Japan."},{"key":"ref_19","unstructured":"Briere, S., Valin, J.M., Michaud, F., and Letourneau, D. (May, January 19\u2013). Embedded Auditory System for Small Mobile Robots. Pasadena, CA, USA."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Valin, J.M., Michaud, F., and Rouat, J. (May, January 14\u2013). Robust 3D Localization and Tracking of Sound Sources Using Beamforming and Particle Filtering. Toulouse, France. Volume 4.","DOI":"10.1109\/ICASSP.2006.1661100"},{"key":"ref_21","unstructured":"Briere, S., L\u00e9tourneau, D., and Fr\u00e9chette, M. (October, January 13\u2013). Embedded and Integration Audition for a Mobile Robot. Washington, DC, USA."},{"key":"ref_22","unstructured":"Nakadai, K., Okuno, H.G., Nakajima, H., Hasegawa, Y., and Tsujino, H. (December, January 1\u2013). An Open Source Software System for Robot Audition HARK and Its Evaluation. Daejeon, South Korea."},{"key":"ref_23","unstructured":"Takahashi, T., Nakadai, K., Komatani, K., Ogata, T., and Okuno, H. (May, January 3\u2013). Improvement in Listening Capability for Humanoid Robot HRP-2. Anchorage, AK, USA."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1016\/j.robot.2006.08.004","article-title":"Robust localization and tracking of simultaneous moving sound sources using beamforming and particle filtering","volume":"55","author":"Valin","year":"2007","journal-title":"Robot. Auton. Syst."},{"key":"ref_25","unstructured":"Hall, E. (1966). The Hidden Dimension, Anchor."},{"key":"ref_26","unstructured":"Argyle, M. (1988). Bodily Communication, Methuen."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"289","DOI":"10.2307\/2786027","article-title":"Eye-contact, distance and affiliation","volume":"28","author":"Argyle","year":"1965","journal-title":"Sociometry"},{"key":"ref_28","unstructured":"Lambert, D. (2004). Body Language, Harper Collins."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1037\/0033-2909.85.1.117","article-title":"Personal space: An evaluative and orienting overview","volume":"85","author":"Hayduk","year":"1978","journal-title":"Psychol. Bull."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Burgoon, J.K., and Jones, S.B. (1976). Toward a Theory of Personal Space Expectations and Their Violations, Human Communication Research.","DOI":"10.1111\/j.1468-2958.1976.tb00706.x"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Nass, C., Steuer, J., and Tauber, E.R. (1994, January 24\u201328). Computers are Social Actors. Boston, MA, USA.","DOI":"10.1145\/259963.260288"},{"key":"ref_32","unstructured":"Takayama, L., and Pantofaru, C. (October, January 11\u2013). Influences on Proxemic Behaviors in Human-Robot Interaction. St. Louis, MO, USA."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1177\/105971230000800104","article-title":"Infant-like social interactions between a robot and a human caregiver","volume":"8","author":"Breazeal","year":"2000","journal-title":"Adapt. Behav."},{"key":"ref_34","unstructured":"H\u00fcttenrauch, H. (October, January 9\u2013). Investigating Spatial Relationships in Human-Robot Interaction. Beijing, China."},{"key":"ref_35","unstructured":"Walters, M. (April, January 6\u2013). An Empirical Framework for Human-Robot Proxemics. Edinburgh, UK. Volume 8."},{"key":"ref_36","unstructured":"Kheng Lee, K., Syrdal, D., Walters, M., and Dautenhahn, K. (August, January 26\u2013). Living with Robots: Investigating the Habituation Effect in Participants' Preferences During a Longitudinal Human-Robot Interaction Study."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Mumm, J., and Mutlu, B. (2011, January 6\u20139). Human-Robot Proxemics. Lausanne, Switzerland.","DOI":"10.1145\/1957656.1957786"},{"key":"ref_38","unstructured":"Henkel, Z. (2010, January 11\u201313). Towards A Computational Method of Scaling A Robot's Behavior via Proxemics. Arlington, VA, USA."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Salichs, M., Barber, R., Khamis, A., Malfaz, M., Gorostiza, J., Pacheco, R., Rivas, R., Corrales, A., Delgado, E., and Garcia, D. (June, January 1\u2013). Maggie: A Robotic Platform for Human-Robot Social Interaction, Bangkok, Thailand.","DOI":"10.1109\/RAMECH.2006.252754"},{"key":"ref_40","unstructured":"Barber, R., and Salichs, M. (2001, January 5\u20137). A New Human Based Architecture for Intelligent Autonomous Robots. Sapporo, Japan."},{"key":"ref_41","unstructured":"Gorostiza, J., Barber, R., Khamis, A., Malfaz, M., Pacheco, R., Rivas, R., Corrales, A., Delgado, E., and Salichs, M. (September, January 6\u2013). Multimodal Human-Robot Interaction Framework for a Personal Robot. Hatfield, UK."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"371","DOI":"10.1007\/s12369-011-0109-8","article-title":"Maggie: A social robot as a gaming platform","volume":"3","author":"Ramey","year":"2011","journal-title":"Int. J. Soc. Robot."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1037\/h0082257","article-title":"Personal space and its relation to extraversion-introversion","volume":"3","author":"Williams","year":"1971","journal-title":"Can. J. Behav. Sci. \/Revue Canadienne des sciences du comportement"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1037\/h0076837","article-title":"Too close for comfort: Sex differences in response to invasions of personal space","volume":"32","author":"Fisher","year":"1975","journal-title":"J. Personal. Soc. Psychol."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Rae, A., Khamis, A., Basir, O., and Kamel, M. (2009, January 6\u20138). Particle filtering for bearing-only audio-visual speaker detection and tracking. Djerba, Tunisia.","DOI":"10.1109\/ICSCS.2009.5412478"},{"key":"ref_46","unstructured":"Falb, J., Rock, T., and Arnautovic, E. (September, January 18\u2013). Using Communicative Acts in Interaction Design Specifications for Automated Synthesis of User Interfaces. Tokyo, Japan."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Falb, J., Popp, R., Rock, T., Jelinek, H., Arnautovic, E., and Kaindl, H. (2007, January 3\u20136). Fully-Automatic Generation of User Interfaces for Multiple Devices from a High-Level Model Based on Communicative Acts. Big Island, HI, USA.","DOI":"10.1109\/HICSS.2007.236"},{"key":"ref_48","unstructured":"Zaslavsky, A. (October, January 13\u2013). Communicative Acts of Elvin-Enhanced Mobile Agents. Halifax, Canada."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/12\/7\/9913\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T21:51:23Z","timestamp":1760219483000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/12\/7\/9913"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,7,23]]},"references-count":48,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2012,7]]}},"alternative-id":["s120709913"],"URL":"https:\/\/doi.org\/10.3390\/s120709913","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2012,7,23]]}}}