{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T17:02:55Z","timestamp":1778864575069,"version":"3.51.4"},"reference-count":48,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2021,3,22]],"date-time":"2021-03-22T00:00:00Z","timestamp":1616371200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004837","name":"Ministerio de Ciencia e Innovaci\u00f3n","doi-asserted-by":"publisher","award":["DPI2013-47944-C4-2-R"],"award-info":[{"award-number":["DPI2013-47944-C4-2-R"]}],"id":[{"id":"10.13039\/501100004837","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper presents a new architecture that integrates Internet of Things (IoT) devices, service robots, and users in a smart assistive environment. A new intuitive and multimodal interaction system supporting people with disabilities and bedbound patients is presented. This interaction system allows the user to control service robots and devices inside the room in five different ways: touch control, eye control, gesture control, voice control, and augmented reality control. The interaction system is comprised of an assistive robotic arm holding a tablet PC. The robotic arm can place the tablet PC in front of the user. A demonstration of the developed technology, a prototype of a smart room equipped with home automation devices, and the robotic assistive arm are presented. The results obtained from the use of the various interfaces and technologies are presented in the article. The results include user preference with regard to eye-base control (performing clicks, and using winks or gaze) and the use of mobile phones over augmented reality glasses, among others.<\/jats:p>","DOI":"10.3390\/s21062212","type":"journal-article","created":{"date-parts":[[2021,3,22]],"date-time":"2021-03-22T12:55:07Z","timestamp":1616417707000},"page":"2212","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":54,"title":["Smart Assistive Architecture for the Integration of IoT Devices, Robotic Systems, and Multimodal Interfaces in Healthcare Environments"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9873-232X","authenticated-orcid":false,"given":"Alberto","family":"Brunete","sequence":"first","affiliation":[{"name":"Centre for Automation and Robotics (CAR UPM-CSIC), Universidad Polit\u00e9cnica de Madrid, 28006 Madrid, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1705-1800","authenticated-orcid":false,"given":"Ernesto","family":"Gambao","sequence":"additional","affiliation":[{"name":"Centre for Automation and Robotics (CAR UPM-CSIC), Universidad Polit\u00e9cnica de Madrid, 28006 Madrid, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9997-0266","authenticated-orcid":false,"given":"Miguel","family":"Hernando","sequence":"additional","affiliation":[{"name":"Centre for Automation and Robotics (CAR UPM-CSIC), Universidad Polit\u00e9cnica de Madrid, 28006 Madrid, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4361-4331","authenticated-orcid":false,"given":"Raquel","family":"Cedazo","sequence":"additional","affiliation":[{"name":"Department of Electrical, Electronical and Automatic Control Engineering and Applied Physics, Escuela T\u00e9cnica Superior de Ingenier\u00eda y Dise\u00f1o Industrial, Universidad Polit\u00e9cnica de Madrid, 28012 Madrid, Spain"}]}],"member":"1968","published-online":{"date-parts":[[2021,3,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Raffaeli, L., Montanini, L., Gambi, E., and Spinsante, S. (2016). User interfaces in smart assistive environments: Requirements, devices, applications. Handbook of Research on Human-Computer Interfaces, Developments, and Applications, IGI Global.","DOI":"10.4018\/978-1-5225-0435-1.ch017"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Bui, H., and Chong, N.Y. (2018, January 27\u201329). An Integrated Approach to Human-Robot-Smart Environment Interaction Interface for Ambient Assisted Living. Proceedings of the IEEE Workshop on Advanced Robotics and its Social Impacts (ARSO), Genova, Italy.","DOI":"10.1109\/ARSO.2018.8625821"},{"key":"ref_3","unstructured":"Sacristan, M., Brunete, A., Hernando, M., and Gambao, E. (2018, January 5). Robohealth: Smart Room Interface for Hospital Patients. Proceedings of the Robots for Assisted Living Workshop, IEEE\/RSJ International Conference on Intelligent Robots and Systems, Madrid, Spain."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1780","DOI":"10.1109\/JPROC.2004.835378","article-title":"Effects of robot-assisted activity for elderly people and nurses at a day service center","volume":"92","author":"Wada","year":"2004","journal-title":"Proc. IEEE"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Chang, W., and \u0160abanovic, S. (2013, January 3\u20136). Potential use of robots in Taiwanese nursing homes. Proceedings of the 8th ACM\/IEEE International Conference on Human-Robot Interaction (HRI), Tokyo, Japan.","DOI":"10.1109\/HRI.2013.6483520"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Park, C., Kang, S., Kim, J., and Oh, J. (2012, January 26\u201328). A study on service robot system for elder care. Proceedings of the 9th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), Daejeon, Korea.","DOI":"10.1109\/URAI.2012.6463070"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Stojmenski, A., Joksimoski, B., Chorbev, I., and Trajkovikj, V. (2016, January 8\u201310). Smart home environment aimed for people with physical disabilities. Proceedings of the IEEE 12th International Conference on Intelligent Computer Communication and Processing (ICCP), Cluj-Napoca, Romania.","DOI":"10.1109\/ICCP.2016.7737115"},{"key":"ref_8","unstructured":"Broxvall, M., Gritti, M., ASaffiotti, A., Seo, B.-S., and Cho, Y.-J. (2006, January 15\u201319). PEIS Ecology: Integrating robots into smart environments. Proceedings of the 2006 IEEE International Conference on Robotics and Automation, Orlando, FL, USA."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1109\/TASE.2007.911674","article-title":"A Steward Robot for Human-Friendly Human-Machine Interaction in a Smart House Envi-ronment","volume":"5","author":"Park","year":"2008","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Cavallo, F., Aquilano, M., Bonaccorsi, M., Limosani, R., Manzi, A., Carrozza, M.C., and Dario, P. (2013, January 6\u201310). On the design, development and experimentation of the ASTRO assistive robot integrated in smart environments. Proceedings of the 2013 IEEE International Conference on Robotics and Automation, Karlsruhe, Germany.","DOI":"10.1109\/ICRA.2013.6631187"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1109\/MPRV.2010.57","article-title":"Echonet: A Home Network Standard","volume":"9","author":"Matsumoto","year":"2010","journal-title":"IEEE Pervasive Comput."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Anghel, I., Cioara, T., Moldovan, D., Antal, M., Pop, C.D., Salomie, I., Pop, C.B., and Chifu, V.R. (2020). Smart Environments and Social Ro-bots for Age-Friendly Integrated Care Services. Int. J. Environ. Res. Public Health, 17.","DOI":"10.3390\/ijerph17113801"},{"key":"ref_13","first-page":"53","article-title":"Avances en el desarrollo de un prototipo de robot asistencial para personas con limitaciones de mo-vilidad","volume":"4","author":"Marquez","year":"2013","journal-title":"Ingenio Magno"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Uehara, T.S.H., and Higa, H. (2010, January 26\u201329). A mobile robotic arm for people with severe disabilities. Proceedings of the 3rd IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics, University of Tokyo, Tokyo, Japan.","DOI":"10.1109\/BIOROB.2010.5626937"},{"key":"ref_15","first-page":"1","article-title":"Robot arm utilized having meal support system based on computer input by human eyes only","volume":"2","author":"Arai","year":"2011","journal-title":"J. Hum. Comput. Interact. (IJHCI)"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1023\/A:1024449018826","article-title":"Development of a robotic arm for handicapped people: A task-oriented design approach","volume":"15","author":"Chang","year":"2003","journal-title":"Auton. Robot."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Ali, S., Madariaga, M.G., McGeary, D.C., Pruehsner, W.R., and Enderle, J.D. (2007, January 10\u201311). The Assistive Robotic Arm. Proceedings of the 33rd Annual Northeast Bioengineering Conference, Stony Brook, NY, USA.","DOI":"10.1109\/NEBC.2007.4413392"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1109\/MRA.2012.2225473","article-title":"Lending a helping hand: Toward novel assistive robotic arms","volume":"20","author":"Groothuis","year":"2013","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_19","unstructured":"Mason, F., Pluchino, P., Cisotto, G., Orso, V., and Gamberini, L. (2020). Internet of Things for Elderly and Fragile People. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1109\/MCOM.001.1900349","article-title":"Requirements and Enablers of Advanced Healthcare Services over Future Cellular Systems","volume":"58","author":"Cisotto","year":"2020","journal-title":"IEEE Commun. Mag."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"853","DOI":"10.1109\/5.664275","article-title":"Toward multimodal human-computer interface","volume":"86","author":"Sharma","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_22","unstructured":"Raisamo, R. (1999). Multimodal Human-Computer Interaction: A Constructive and Empirical Study, Tampere University Press."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Gyurian, N., Drozd, I., and Rozinaj, G. (2016, January 12\u201314). Multimodal interface for smart home. Proceedings of the International Symposium ELMAR, Zadar, Croatia.","DOI":"10.1109\/ELMAR.2016.7731806"},{"key":"ref_24","unstructured":"Jeon, Y., and Ahn, H. (2011, January 23\u201326). A multimodal ubiquitous interface system using smart phone for human-robot interaction. Proceedings of the 8th Interna-tional Conference on Ubiquitous Robots and Ambient Intelligence (URAI), Incheon, Korea."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Maimon-Mor, R.O., Fernandez-Quesada, J., Zito, G.A., Konnaris, C., Dziemian, S., and Faisal, A.A. (2017, January 17\u201320). Towards free 3D end-point control for robotic-assisted human reaching using binocular eye tracking. Proceedings of the International Conference on Rehabilitation Robotics (ICORR), London, UK.","DOI":"10.1109\/ICORR.2017.8009388"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Tostado, P.M., Abbott, W.W., and Faisal, A.A. (2016, January 16\u201321). 3D gaze cursor: Continuous calibration and end-point grasp control of robotic actuators. Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden.","DOI":"10.1109\/ICRA.2016.7487502"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Mtshali, P., and Khubisa, F. (2019, January 6\u20138). A Smart Home Appliance Control System for Physically Disabled People. Proceedings of the Conference on Information Communications Technology and Society (ICTAS), Durban, South Africa.","DOI":"10.1109\/ICTAS.2019.8703637"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"K\u00ebpuska, V., and Bohouta, G. (2018, January 8\u201310). Next-generation of virtual personal assistants (Microsoft Cortana, Apple Siri, Amazon Alexa and Google Home). Proceedings of the IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, USA.","DOI":"10.1109\/CCWC.2018.8301638"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Nasr, M., Karray, F., and Quintana, Y. (2020, January 11\u201314). Human Machine Interaction Platform for Home Care Support System. Proceedings of the IEEE International Conference on Systems, Man and Cybernetics (SMC), Toronto, ON, Canada.","DOI":"10.1109\/SMC42975.2020.9283095"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Elleuch, H., Wali, A., and Alimi, A.M. (2014, January 27\u201329). Smart Tablet Monitoring by a Real-Time Head Movement and Eye Gestures Recognition Sys-tem. Proceedings of the International Conference on Future Internet of Things and Cloud, Barcelona, Spain.","DOI":"10.1109\/FiCloud.2014.70"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1109\/TCYB.2013.2252792","article-title":"A Low-Computational Approach on Gaze Estimation With Eye Touch System","volume":"44","author":"Topal","year":"2014","journal-title":"IEEE Trans. Cybern."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Noronha, B., Dziemian, S., Zito, G.A., Konnaris, C., and Faisal, A.A. (2017, January 17\u201320). Wink to grasp: Comparing eye, voice and EMG gesture control of grasp with soft-robotic gloves. Proceedings of the International Conference on Rehabilitation Robotics (ICORR), London, UK.","DOI":"10.1109\/ICORR.2017.8009387"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Sushmit, S., Haque, F.M., Shahriar, M., Bhuiyan, S.A.M., and Sarkar, M.A.R. (2017, January 21\u201322). Design of a gesture controlled robotic gripper arm using neural networks. Proceedings of the IEEE International Conference on Power, Control, Signals and Instrumentation Engineering (ICPCSI), Chennai, India.","DOI":"10.1109\/ICPCSI.2017.8392180"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"564","DOI":"10.1109\/TBCAS.2018.2810256","article-title":"A Multimodal Adaptive Wireless Control Interface for People with Upper-Body Disabilities","volume":"12","author":"Fall","year":"2018","journal-title":"IEEE Trans. Biomed. Circ. Syst."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Shin, S., Kim, D., and Seo, Y. (2014, January 8\u201310). Controlling Mobile Robot Using IMU and EMG Sensor-Based Gesture Recognition. Proceedings of the Ninth International Conference on Broadband and Wireless Computing, Communication and Applications, Guangdong, China.","DOI":"10.1109\/BWCCA.2014.145"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Yu, Y., Wang, X., Zhong, Z., and Zhang, Y. (2017, January 28\u201330). ROS-based UAV control using hand gesture recognition. Proceedings of the 29th Chinese Control and Decision Conference (CCDC), Chongqing, China.","DOI":"10.1109\/CCDC.2017.7978402"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Lamb, K., and Madhe, S. (2016, January 9\u201311). Hand gesture recognition based bed position control for disabled patients. Proceedings of the Conference on Advances in Signal Processing (CASP), Pune, India.","DOI":"10.1109\/CASP.2016.7746159"},{"key":"ref_38","unstructured":"Ransalu, S., and Kumarawadu, S. (2012, January 12\u201315). A robust vision-based hand gesture recognition system for appliance control in smart homes. Proceedings of the IEEE International Conference on Signal Processing, Communication and Computing (ICSPCC 2012), Hong Kong, China."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Handosa, M., Gracanin, D., Elmongui, H.G., and Ciambrone, A. (2017, January 18\u201319). Painting with light: Gesture based light control in architectural set-tings. Proceedings of the IEEE Symposium on 3D User Interfaces (3DUI), Los Angeles, CA, USA.","DOI":"10.1109\/3DUI.2017.7893365"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Czuszynski, K., and Ruminski, J. (2018, January 21\u201323). Towards Contactless, Hand Gestures-Based Control of Devices. Proceedings of the 44th Annual Conference of the IEEE Industrial Electronics Society, Washington, DC, USA.","DOI":"10.1109\/IECON.2018.8591446"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Petersen, N., and Stricker, D. (2009, January 19\u201322). Continuous natural user interface: Reducing the gap between real and digital world. Proceedings of the 2009 IEEE International Symposium on Mixed and Augmented Reality, Orlando, FL, USA.","DOI":"10.1109\/ISMAR.2009.5336502"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1167\/iovs.04-1279","article-title":"Asymmetry of blinking","volume":"47","author":"Kassem","year":"2006","journal-title":"Investig. Ophthalmol. Vis. Sci."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"779","DOI":"10.1007\/s12369-019-00614-y","article-title":"ENRICHME: Perception and Interaction of an Assistive Robot for the Elderly at Home","volume":"12","author":"Agrigoroaie","year":"2020","journal-title":"Int. J. Soc. Robot."},{"key":"ref_44","first-page":"1","article-title":"Exploring Eye-Gaze Wheelchair Control","volume":"Volume 16","author":"Araujo","year":"2020","journal-title":"ACM Symposium on Eye Tracking Research and Applications (ETRA \u201920 Adjunct)"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Rub\u00ed, S., Jes\u00fas, N., Gondim, L., and Paulo, R. (2019). IoMT Platform for Pervasive Healthcare Data Aggregation, Processing, and Sharing Based on OneM2M and OpenEHR. Sensors, 19.","DOI":"10.3390\/s19194283"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Rincon, J.A., Guerra-Ojeda, S., Carrascosa, C., and Julian, V. (2020). An IoT and Fog Computing-Based Monitoring System for Cardio-vascular Patients with Automatic ECG Classification Using Deep Neural Networks. Sensors, 20.","DOI":"10.3390\/s20247353"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Al-khafajiy, M., Webster, L., Baker, T., and Waraich, A. (2018, January 26\u201327). Towards fog driven IoT healthcare: Challenges and framework of fog computing in healthcare. Proceedings of the 2nd International Conference on Future Networks and Distributed Systems (ICFNDS \u201918). Association for Computing Machinery, New York, NY, USA.","DOI":"10.1145\/3231053.3231062"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Cipresso, P., Serino, S., and Villani, D. (2019). System of Nudge Theory-Based ICT Applications for Older Citizens: The SENIOR Project. Pervasive Computing Paradigms for Mental Health, Springer. MindCare 2019; Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering.","DOI":"10.1007\/978-3-030-25872-6"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/6\/2212\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:39:01Z","timestamp":1760161141000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/6\/2212"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,22]]},"references-count":48,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2021,3]]}},"alternative-id":["s21062212"],"URL":"https:\/\/doi.org\/10.3390\/s21062212","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,22]]}}}