{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,13]],"date-time":"2025-11-13T12:37:24Z","timestamp":1763037444522,"version":"build-2065373602"},"reference-count":36,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2020,4,22]],"date-time":"2020-04-22T00:00:00Z","timestamp":1587513600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"European Regional Development Fund, Regional Operational Programme of the Opolskie region for the years 2014-2020: Priority Axis 01 Innovation in the Economy, Measure 1.1 Innovation in Enterprises","award":["RPOP.01.01.00-16-0072\/16-00"],"award-info":[{"award-number":["RPOP.01.01.00-16-0072\/16-00"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper presents a more detailed concept of Human-Robot Interaction systems architecture. One of the main differences between the proposed architecture and other ones is the methodology of information acquisition regarding the robot\u2019s interlocutor. In order to obtain as much information as possible before the actual interaction took place, a custom Internet-of-Things-based sensor subsystems connected to Smart Infrastructure was designed and implemented, in order to support the interlocutor identification and acquisition of initial interaction parameters. The Artificial Intelligence interaction framework of the developed robotic system (including humanoid Pepper with its sensors and actuators, additional local, remote and cloud computing services) is being extended with the use of custom external subsystems for additional knowledge acquisition: device-based human identification, visual identification and audio-based interlocutor localization subsystems. These subsystems were deeply introduced and evaluated in this paper, presenting the benefits of integrating them into the robotic interaction system. In this paper a more detailed analysis of one of the external subsystems\u2014Bluetooth Human Identification Smart Subsystem\u2014was also included. The idea, use case, and a prototype, integration of elements of Smart Infrastructure systems and the prototype implementation were performed in a small front office of the Weegree company as a decent test-bed application area.<\/jats:p>","DOI":"10.3390\/s20082376","type":"journal-article","created":{"date-parts":[[2020,4,23]],"date-time":"2020-04-23T02:10:52Z","timestamp":1587607852000},"page":"2376","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":35,"title":["Human Interaction Smart Subsystem\u2014Extending Speech-Based Human-Robot Interaction Systems with an Implementation of External Smart Sensors"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1080-6767","authenticated-orcid":false,"given":"Michal","family":"Podpora","sequence":"first","affiliation":[{"name":"Faculty of Electrical Engineering, Automatic Control and Informatics\u2014Opole University of Technology, 45-758 Opole, Poland"},{"name":"Weegree Sp. z o.o. S.K., 45-018 Opole, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4420-1157","authenticated-orcid":false,"given":"Arkadiusz","family":"Gardecki","sequence":"additional","affiliation":[{"name":"Faculty of Electrical Engineering, Automatic Control and Informatics\u2014Opole University of Technology, 45-758 Opole, Poland"},{"name":"Weegree Sp. z o.o. S.K., 45-018 Opole, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9554-489X","authenticated-orcid":false,"given":"Ryszard","family":"Beniak","sequence":"additional","affiliation":[{"name":"Faculty of Electrical Engineering, Automatic Control and Informatics\u2014Opole University of Technology, 45-758 Opole, Poland"},{"name":"Weegree Sp. z o.o. S.K., 45-018 Opole, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0577-7903","authenticated-orcid":false,"given":"Bartlomiej","family":"Klin","sequence":"additional","affiliation":[{"name":"Weegree Sp. z o.o. S.K., 45-018 Opole, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3574-4697","authenticated-orcid":false,"given":"Jose Lopez","family":"Vicario","sequence":"additional","affiliation":[{"name":"Wireless Information Networking Group, Escola d\u2019Enginyeria\u2014Universitat Autonoma de Barcelona, 08193 Bellaterra, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7826-1292","authenticated-orcid":false,"given":"Aleksandra","family":"Kawala-Sterniuk","sequence":"additional","affiliation":[{"name":"Faculty of Electrical Engineering, Automatic Control and Informatics\u2014Opole University of Technology, 45-758 Opole, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,4,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1016\/j.ifacol.2019.12.685","article-title":"Humanoid Receptionist Connected to IoT Subsystems and Smart Infrastructure is Smarter than Expected","volume":"52","author":"Podpora","year":"2019","journal-title":"IFAC-PapersOnLine"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"138","DOI":"10.1016\/j.ifacol.2018.07.143","article-title":"Innovative Internet of Things-reinforced Human Recognition for Human-Machine Interaction Purposes","volume":"51","author":"Gardecki","year":"2018","journal-title":"IFAC-PapersOnLine"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Redondo, M.E.L. (2019, January 3\u20136). Comfortability Detection for Adaptive Human-Robot Interactions. Proceedings of the 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW), Cambridge, UK.","DOI":"10.1109\/ACIIW.2019.8925017"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Belanche, D., Casalo, L.V., and Flavian, C. (2020). Customer\u2019s Acceptance of Humanoid Robots in Services: The Moderating role of Risk Aversion. Marketing and Smart Technologies, Springer.","DOI":"10.1007\/978-981-15-1564-4_42"},{"key":"ref_5","unstructured":"Cambridge Centre for Smart Infrastructure and Construction (2020, April 01). What Is Smart Infrastructure?. Available online: https:\/\/iotevent.eu\/what-is-smart-infrastructure\/."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Sahu, S.N., Moharana, M., Prusti, P.C., Chakrabarty, S., Khan, F., and Pattanayak, S.K. (2020). Real-Time Data Analytics in Healthcare Using the Internet of Things. Real-Time Data Analytics for Large Scale Sensor Data, Academic Press.","DOI":"10.1016\/B978-0-12-818014-3.00002-4"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Greco, A., Roberto, A., Saggese, A., Vento, M., and Vigilante, V. (2019, January 6\u20139). Emotion Analysis from Faces for Social Robotics. Proceedings of the IEEE International Conference on Systems, Man and Cybernetics (SMC), Bari, Italy.","DOI":"10.1109\/SMC.2019.8914039"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1007\/s10846-018-0967-5","article-title":"A design and analysis methodology for component-based real-time architectures of autonomous systems","volume":"96","author":"Gobillot","year":"2019","journal-title":"J. Intell. Rob. Syst."},{"key":"ref_9","first-page":"22","article-title":"Humanlike robots as employees in the hotel industry: Thematic content analysis of online reviews","volume":"29","author":"Yu","year":"2020","journal-title":"J. Hosp. Mark. Manag."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"102501","DOI":"10.1016\/j.ijhm.2020.102501","article-title":"Human staff vs. service robot vs. fellow customer: Does it matter who helps your customer following a service failure incident?","volume":"87","author":"Ho","year":"2020","journal-title":"Int. J. Hosp. Manag."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Lu, V.N., Wirtz, J., Kunz, W.H., Paluch, S., Gruber, T., Martins, A., and Patterson, P.G. (2020). Service robots, customers, and service employees: What can we learn from the academic literature and where are the gaps?. J. Serv. Theory Pract.","DOI":"10.2139\/ssrn.3806199"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Hoffman, G., and Breazeal, C. (2008, January 12\u201315). Achieving Fluency through Perceptual-Symbol Practice in Human-Robot Collaboration. Proceedings of the 3rd ACM\/IEEE International Conference on Human-Robot Interaction (HRI), Amsterdam, The Netherlands.","DOI":"10.1145\/1349822.1349824"},{"key":"ref_13","unstructured":"Kreps, J., Narkhede, N., and Rao, J. (2011, January 12\u201316). Kafka: A Distributed Messaging System for Log Processing. Proceedings of the NetDB, Athens, Greece."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Saeedvand, S., Jafari, M., Aghdasi, H.S., and Baltes, J. (2019). A comprehensive survey on humanoid robot development. Knowl. Eng. Rev., 34.","DOI":"10.1017\/S0269888919000158"},{"key":"ref_15","unstructured":"Bergkvist, A., Burnett, D.C., Jennings, C., Narayanan, A., Aboba, B., Brandstetter, T., and Bruaroey, J.I. (2018). WebRTC 1.0: Real-Time Communication between Browsers. W3C Candidate Recomm., Available online: https:\/\/www.w3.org\/TR\/2018\/CR-webrtc-20180927\/."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Andr\u00e9, E., Le Breton, N., Lemesle, A., Roux, L., and Gouaillard, A. (2018, January 16\u201318). Comparative Study of WebRTC Open Source SFUs for Video Conferencing. Proceedings of the Principles, Systems and Applications of IP Telecommunications (IPTComm), Chicago, IL, USA.","DOI":"10.1109\/IPTCOMM.2018.8567642"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1145\/3199524.3199534","article-title":"Performance evaluation of WebRTC-based video conferencing","volume":"45","author":"Jansen","year":"2018","journal-title":"ACM SIGMETRICS Perform. Eval. Rev."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Akhtar, M.T., and Mitsuhashi, W. (2009, January 24\u201327). Improving Performance of Active Noise Control Systems in the Presence of Uncorrelated Periodic Disturbance at Error Microphone. Proceedings of the IEEE Int Symposium on Circuits and Systems, Taipei, Taiwan.","DOI":"10.1109\/ISCAS.2009.5118194"},{"key":"ref_19","unstructured":"Freiberger, K., Huber, S., and Meerwald, P. (2013, January 9\u201312). Multi-Channel Noise\/Echo Reduction in PulseAudio on Embedded Linux. Proceedings of the Linux Audio Conference, Graz, Austria."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Gardecki, A., Podpora, M., Beniak, R., and Klin, B. (2018, January 18\u201322). The Pepper Humanoid Robot in Front Desk Application. Proceedings of the Progress in Applied Electrical Engineering (PAEE), Koscielisko, Poland.","DOI":"10.1109\/PAEE.2018.8441069"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Yan, L., Zhang, Y., Yang, L.T., and Ning, H. (2008). The Internet of Things: From RFID to the Next-Generation Pervasive Networked Systems, CRC Press.","DOI":"10.1201\/9781420052824"},{"key":"ref_22","unstructured":"SoftBank Robotics (2020, April 01). Who Is Pepper?. Available online: https:\/\/www.ald.softbankrobotics.com\/en\/cool-robots\/pepper."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Barra, P., Bisogni, C., Rapuano, A., Abate, A.F., and Iovane, G. (2019, January 5\u20138). HiMessage: An Interactive Voice Mail System with the Humanoid Robot Pepper. Proceedings of the IEEE International Conference on Dependable, Autonomic and Secure Computing, International Conference on Pervasive Intelligence and Computing, International Conference on Cloud and Big Data Computing, International Conference on Cyber Science and Technology Congress (DASC\/PiCom\/CBDCom\/CyberSciTech), Fukuoka, Japan.","DOI":"10.1109\/DASC\/PiCom\/CBDCom\/CyberSciTech.2019.00123"},{"key":"ref_24","unstructured":"SoftBank Robotics (2018, April 11). SoftBank Guidelines Documentation. Available online: http:\/\/doc.aldebaran.com\/download\/Pepper_B2BD_guidelines_Sept_V1.5.pdf."},{"key":"ref_25","unstructured":"Forbrig, P. (2019, December 01). Challenges in Multi\u2013User Interaction with a Social Humanoid Robot Pepper. Available online: http:\/\/ceur-ws.org\/Vol-2503\/paper1_2.pdf."},{"key":"ref_26","unstructured":"Metageek (2020, March 01). Understanding RSSI. Available online: https:\/\/www.metageek.com\/training\/resources\/understanding-rssi.html."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Ruijten, P.A., and Cuijpers, R.H. (2020). Do Not Let the Robot Get too Close: Investigating the Shape and Size of Shared Interaction Space for Two People in a Conversation. Information, 11.","DOI":"10.3390\/info11030147"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1003","DOI":"10.1525\/aa.1963.65.5.02a00020","article-title":"A system for the notation of proxemic behavior","volume":"65","author":"Hall","year":"1963","journal-title":"Am. Anthropol."},{"key":"ref_29","unstructured":"Mest, E. (2016). How Technology and Robotics Are Changing the Front Desk, Hotel Management."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Tussyadiah, P., and Park, S. (2016). Consumer Evaluation of Hotel Service Robots. Information and Communication Technologies in Tourism, Springer.","DOI":"10.1007\/978-3-319-72923-7_24"},{"key":"ref_31","first-page":"247","article-title":"Enhancing hospitality experience with service robots: The mediating role of rapport building","volume":"29","author":"Qiu","year":"2020","journal-title":"J. Hosp. Mark. Manag."},{"key":"ref_32","unstructured":"Rozanska, A., and Podpora, M. (2018, January 13\u201314). Making Eye Contact with Robot\u2014Exploring User Experience in Interacting with Pepper. Proceedings of the Advances in Intelligent Systems and Computing Book Series (AISC), Opole, Poland."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Robinette, P., Li, W., Allen, R., Howard, A.M., and Wagner, A.R. (2016, January 7\u201310). Overtrust of Robots in Emergency Evacuation Scenarios. Proceedings of the ACM\/IEEE International Conference on Human-Robot Interaction (HRI), Christchurch, New Zealand.","DOI":"10.1109\/HRI.2016.7451740"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Chung, M.J.-Y., and Cakmak, M. (2018, January 5\u20138). Exploring the Use of Robots for Gathering Customer Feedback in the Hospitality Industry. Proceedings of the HRI Workshop on Social Robots in the Wild, Chicago, IL, USA.","DOI":"10.1109\/ROMAN.2018.8525604"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"105613","DOI":"10.1016\/j.asoc.2019.105613","article-title":"A consensus novelty detection ensemble approach for anomaly detection in activities of daily living","volume":"83","author":"Yahaya","year":"2019","journal-title":"Appl. Soft Comput."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Yahaya, S.W., Lotfi, A., Mahmud, M., Machado, P., and Kubota, N. (2019, January 6\u20139). Gesture Recognition Intermediary Robot for Abnormality Detection in Human Activities. Proceedings of the IEEE Symposium Series on Computational Intelligence, Xiamen, China.","DOI":"10.1109\/SSCI44817.2019.9003121"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/8\/2376\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T14:08:59Z","timestamp":1760364539000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/8\/2376"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,4,22]]},"references-count":36,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2020,4]]}},"alternative-id":["s20082376"],"URL":"https:\/\/doi.org\/10.3390\/s20082376","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2020,4,22]]}}}