{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,28]],"date-time":"2025-11-28T12:31:37Z","timestamp":1764333097746,"version":"build-2065373602"},"reference-count":83,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2021,11,1]],"date-time":"2021-11-01T00:00:00Z","timestamp":1635724800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001871","name":"Funda\u00e7\u00e3o para a Ci\u00eancia e Tecnologia","doi-asserted-by":"publisher","award":["UID\/CEC\/00127\/2021"],"award-info":[{"award-number":["UID\/CEC\/00127\/2021"]}],"id":[{"id":"10.13039\/501100001871","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Compete 2020, Lisboa 2020 and Portugal 2020","award":["POCI-01-0247-FEDER-046115"],"award-info":[{"award-number":["POCI-01-0247-FEDER-046115"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IJERPH"],"abstract":"<jats:p>This study aimed to determine the methodological quality of user-centered usability evaluation of Ambient Assisted Living (AAL) solutions by (i) identifying the characteristics of the AAL studies reporting on user-centered usability evaluation, (ii) systematizing the methods, procedures and instruments being used, and (iii) verifying if there is evidence of a common understanding on methods, procedures, and instruments for user-centered usability evaluation. An electronic search was conducted on Web of Science, Scopus, and IEEE Xplore databases, combining relevant keywords. Then, titles and abstracts were screened against inclusion and exclusion criteria, and the full texts of the eligible studies were retrieved and screened for inclusion. A total of 44 studies were included. The results show a great heterogeneity of methods, procedures, and instruments to evaluate the usability of AAL solutions and, in general, the researchers fail to consider and report relevant methodological aspects. Guidelines and instruments to assess the quality of the studies might help improving the experimental design and reporting of studies on user-centered usability evaluation of AAL solutions.<\/jats:p>","DOI":"10.3390\/ijerph182111507","type":"journal-article","created":{"date-parts":[[2021,11,1]],"date-time":"2021-11-01T22:21:34Z","timestamp":1635805294000},"page":"11507","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Methodological Quality of User-Centered Usability Evaluation of Ambient Assisted Living Solutions: A Systematic Literature Review"],"prefix":"10.3390","volume":"18","author":[{"given":"Rute","family":"Bastardo","sequence":"first","affiliation":[{"name":"UNIDCOM, Science and Technology School, University of Tr\u00e1s-os-Montes and Alto Douro, Quinta de Prado, 5001-801 Vila Real, Portugal"}]},{"given":"Ana Isabel","family":"Martins","sequence":"additional","affiliation":[{"name":"Center for Health Technology and Services Research, Health Sciences School, University of Aveiro, 3810-193 Aveiro, Portugal"}]},{"given":"Jo\u00e3o","family":"Pav\u00e3o","sequence":"additional","affiliation":[{"name":"INESC-TEC, Science and Technology School, University of Tr\u00e1s-os-Montes and Alto Douro, Quinta de Prado, 5001-801 Vila Real, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4386-5851","authenticated-orcid":false,"given":"Anabela Gon\u00e7alves","family":"Silva","sequence":"additional","affiliation":[{"name":"Center for Health Technology and Services Research, Health Sciences School, University of Aveiro, 3810-193 Aveiro, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3801-7249","authenticated-orcid":false,"given":"Nelson Pacheco","family":"Rocha","sequence":"additional","affiliation":[{"name":"Department of Medical Sciences, IEETA-Institute of Electronics and Informatics Engineering of Aveiro, University of Aveiro, 3810-193 Aveiro, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2021,11,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1007\/s12126-008-9019-y","article-title":"Ageing in Place in the United Kingdom","volume":"32","author":"Sixsmith","year":"2008","journal-title":"Ageing Int."},{"key":"ref_2","unstructured":"World Health Organization (2002). Active Ageing: A policy Framework, WHO."},{"key":"ref_3","unstructured":"World Health Organization (2004). A Glossary of Terms for Community Health Care and Services for Older Persons, WHO."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1650","DOI":"10.1016\/j.socscimed.2005.08.044","article-title":"Behavioral science at the crossroads in public health: Extending horizons, envisioning the future","volume":"62","author":"Glass","year":"2006","journal-title":"Soc. Sci. Med."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Lindgren, H., Baskar, J., Guerrero, E., Nieves, J.C., Nilsson, I., and Yan, C. (2016, January 11\u201313). Computer-Supported Assessment for Tailoring Assistive Technology. Proceedings of the 6th International Conference on Digital Health Conference, Montr\u00e9al, QC, Canada.","DOI":"10.1145\/2896338.2896352"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"761","DOI":"10.1007\/s12652-018-0856-6","article-title":"Listening to the ones who care: Exploring the perceptions of informal caregivers towards ambient assisted living applications","volume":"10","author":"Jaschinski","year":"2019","journal-title":"J. Ambient Intell. Humaniz. Comput."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1016\/j.pmcj.2011.10.004","article-title":"Pervasive computing at scale: Transforming the state of the art","volume":"8","author":"Cook","year":"2012","journal-title":"Pervasive Mob. Comput."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1109\/2.237456","article-title":"Hot topics: Ubiquitous computing","volume":"26","author":"Weiser","year":"1993","journal-title":"IEEE Comput."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Augusto, J. (2012). Handbook of Ambient Assisted Living: Technology for Healthcare, Rehabilitation and Well-Being, IOS Press.","DOI":"10.1093\/itnow\/bws037"},{"key":"ref_10","unstructured":"Costa, R., Carneiro, D., Novais, P., and Lima, L. (2008, January 22\u201324). Ambient assisted living. Proceedings of the 3rd Symposium of Ubiquitous Computing and Ambient Intelligence, Salamanca, Spain."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1016\/S0921-8890(98)00083-9","article-title":"Intelligent mobile agents in elderly care","volume":"27","author":"Vieira","year":"1999","journal-title":"Robot. Auton. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"D\u00edaz-Oreiro, I., L\u00f3pez, G., Quesada, L., and Guerrero, L.A. (2021). UX Evaluation with Standardized Questionnaires in Ubiquitous Computing and Ambient Intelligence: A Systematic Literature Review. Adv. Hum. Comput. Interact., 5518722.","DOI":"10.1155\/2021\/5518722"},{"key":"ref_13","unstructured":"International Organization for Standardization (1999). ISO 9241 Part 11\u2014Guidance on Usability, International Organization for Standardization."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Nielsen, J. (1994). Usability Engineering, Morgan Kaufmann.","DOI":"10.1016\/B978-0-08-052029-2.50009-7"},{"key":"ref_15","unstructured":"Bevan, N., Claridge, N., and Petrie, H. (2005, January 22\u201327). Tenuta: Simplified guidance for usability and accessibility. Proceedings of the HCI International 2005, Las Vegas, NV, USA."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"222","DOI":"10.1080\/10645578.2014.945354","article-title":"A review of \u2018universal methods of design: 100 ways to research complex problems, develop innovative ideas, and design effective solutions\u2019","volume":"17","author":"Morrissey","year":"2014","journal-title":"Visit. Stud."},{"key":"ref_17","unstructured":"Dix, A., Finlay, G., Abowd, G., and Beale, R. (2004). Human-Computer Interaction, Prentice Hall."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"116145","DOI":"10.1109\/ACCESS.2019.2910778","article-title":"Set of usability heuristics for quality assessment of mobile applications on smartphones","volume":"7","author":"Canedo","year":"2019","journal-title":"IEEE Access"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Bernsen, N., and Dybkj\u00e6r, L. (2010). Multimodal Usability, Springer.","DOI":"10.1007\/978-1-84882-553-6"},{"key":"ref_20","unstructured":"Mahmood, Z., and Saeed, S. (2015). Usability Evaluation Methods: A Systematic Review. Human Factors. Software Development and Design, IGI Global."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Marcos-Pablos, S., and Garc\u00eda-Pe\u00f1alvo, F.J. (2019). Technological ecosystems in care and assistance: A systematic literature review. Sensors, 19.","DOI":"10.3390\/s19030708"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"17","DOI":"10.4018\/IJEHMC.2020010102","article-title":"Assessment of reference architectures and reference models for ambient assisted living systems: Results of a systematic literature review","volume":"11","author":"Oquendo","year":"2020","journal-title":"Int. J. E-Health Med. Commun."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Sanchez-Comas, A., Synnes, K., and Hallberg, J. (2020). Hardware for recognition of human activities: A review of smart home and AAL related technologies. Sensors, 20.","DOI":"10.3390\/s20154227"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Choukou, M.A., Shortly, T., Leclerc, N., Freier, D., Lessard, G., Demers, L., and Auger, C. (2021). Evaluating the acceptance of ambient assisted living technology (AALT) in rehabilitation: A scoping review. Int. J. Med. Inform., 104461.","DOI":"10.1016\/j.ijmedinf.2021.104461"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1016\/j.patrec.2019.03.016","article-title":"Positive technology for elderly well-being: A review","volume":"137","author":"Grossi","year":"2020","journal-title":"Pattern Recognit. Lett."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"e225","DOI":"10.1002\/hsr2.225","article-title":"Ambient assisted living technology-mediated interventions for older people and their informal carers in the context of healthy ageing: A scoping review","volume":"4","author":"Nilsson","year":"2021","journal-title":"Health Sci. Rep."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Puliga, G., Nasullaev, A., Bono, F., Guti\u00e9rrez, E., and Strozzi, F. (2020). Ambient assisted living and European funds: A bibliometric approach. Inf. Technol. People.","DOI":"10.1108\/ITP-10-2019-0565"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"336","DOI":"10.1016\/j.ijsu.2010.02.007","article-title":"Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement","volume":"8","author":"Moher","year":"2009","journal-title":"Int. J. Surg."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1177\/0739456X17723971","article-title":"Guidance on conducting a systematic literature review","volume":"39","author":"Xiao","year":"2019","journal-title":"J. Plan. Educ. Res."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"e14829","DOI":"10.2196\/14829","article-title":"A scale to assess the methodological quality of studies assessing usability of electronic health products and services: Delphi study followed by validity and reliability testing","volume":"21","author":"Silva","year":"2019","journal-title":"J. Med. Internet Res."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Morandell, M.M., Hochgatterer, A., Fagel, S., and Wassertheurer, S. (2008, January 20\u201321). Avatars in assistive homes for the elderly. Proceedings of the Symposium of the Austrian HCI and Usability Engineering Group, Graz, Austria.","DOI":"10.1007\/978-3-540-89350-9_27"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Maier, E., and Kempter, G. (2009, January 19\u201324). AAL in the Wild\u2013Lessons Learned. Proceedings of the International Conference on Universal Access in Human-Computer Interaction, San Diego, CA, USA.","DOI":"10.1007\/978-3-642-02710-9_25"},{"key":"ref_33","first-page":"G27","article-title":"Implementing ambient assisting technologies in elder-care: Results of a pilot study","volume":"2","author":"Hazzam","year":"2011","journal-title":"Synesis J. Sci. Technol. Ethics Policy"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Werner, K., Oberzaucher, J., and Werner, F. (2012, January 4\u20136). Evaluation of human robot interaction factors of a socially assistive robot together with older people. Proceedings of the 6th International Conference on Complex, Intelligent, and Software Intensive Systems, Palermo, Italy.","DOI":"10.1109\/CISIS.2012.36"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"547","DOI":"10.3233\/AIS-130234","article-title":"A personalized exercise trainer for the elderly","volume":"5","author":"Bleser","year":"2013","journal-title":"J. Ambient Intell. Smart Environ."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"954","DOI":"10.1007\/s12559-014-9290-z","article-title":"Development of a socially believable multi-robot solution from town to home","volume":"6","author":"Cavallo","year":"2014","journal-title":"Cogn. Comput."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1108\/JAT-10-2013-0031","article-title":"Expectations and user experience of a multimodal medicine management system for older users","volume":"8","author":"Harjumaa","year":"2014","journal-title":"J. Assist. Technol."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"R\u00f6hrbein, F., Veiga, G., and Natale, C. (2014). Improving domiciliary robotic services by integrating the ASTRO robot in an AmI infrastructure. Gearing up and Accelerating Cross-Fertilization between Academic and Industrial Robotics Research in Europe, Springer.","DOI":"10.1007\/978-3-319-03838-4"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1629","DOI":"10.3390\/s140101629","article-title":"A smart kitchen for ambient assisted living","volume":"14","author":"Blasco","year":"2014","journal-title":"Sensors"},{"key":"ref_40","first-page":"1024","article-title":"AAL@ MEO: Interactive Digital-TV to Support Home Care","volume":"217","author":"Ribeiro","year":"2015","journal-title":"Stud. Health Technol. Inform."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"184","DOI":"10.1007\/s10916-015-0342-z","article-title":"Integration of wearable solutions in aal environments with mobility support","volume":"39","author":"Costa","year":"2015","journal-title":"J. Med. Syst."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Dias, M.S., Vilar, E., Sousa, F., Vasconcelos, A., Pinto, F.M., Saldanha, N., and Eloy, S. (2015, January 2\u20137). A living labs approach for usability testing of Ambient Assisted Living technologies. Proceedings of the 4th International Conference Design, User Experience, and Usability: Design Discourse, Los Angeles, CA, USA.","DOI":"10.1007\/978-3-319-20886-2_17"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3109\/17538157.2013.872114","article-title":"A novel multimodal tool for telemonitoring patients with COPD","volume":"40","author":"Crespo","year":"2015","journal-title":"Inform. Health Soc. Care"},{"key":"ref_44","first-page":"1","article-title":"On the effect of previous technological experience on the usability of a virtual rehabilitation tool for the physical activation and cognitive stimulation of elders","volume":"39","author":"Grimaldo","year":"2015","journal-title":"J. Med. Syst."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1186\/s11556-016-0165-z","article-title":"Exploring user experience and technology acceptance for a fall prevention system: Results from a randomized clinical trial and a living lab","volume":"13","author":"Vaziri","year":"2016","journal-title":"Eur. Rev. Aging Phys. Act."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2967102","article-title":"ICT-based fall prevention system for older adults: Qualitative results from a long-term field study","volume":"23","author":"Ogonowski","year":"2016","journal-title":"ACM Trans. Comput.-Hum. Interact."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1080\/07370024.2015.1085309","article-title":"A kinect and inertial sensor-based system for the self-assessment of fall risk: A home-based study in older people","volume":"31","author":"Ejupi","year":"2016","journal-title":"Hum.-Comput. Interact."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"282","DOI":"10.1007\/s00391-016-1067-4","article-title":"Social service robots to support independent living","volume":"49","author":"Pripfl","year":"2016","journal-title":"Z. Gerontol. Geriatr."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"783","DOI":"10.3233\/AIS-170457","article-title":"Delivering elder-care environments utilizing TV-channel based mechanisms","volume":"9","author":"Amaxilatis","year":"2017","journal-title":"J. Ambient Intell. Smart Environ."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"545","DOI":"10.1007\/s10209-016-0487-7","article-title":"Design and development of Medication Assistant: Older adults centred design to go beyond simple medication reminders","volume":"16","author":"Teixeira","year":"2017","journal-title":"Univers. Access Inf. Soc."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Goumopoulos, C., Papa, I., and Stavrianos, A. (2017). Development and evaluation of a mobile application suite for enhancing the social inclusion and well-being of seniors. Informatics, 4.","DOI":"10.3390\/informatics4030015"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"1263","DOI":"10.1007\/s10514-016-9586-9","article-title":"Enabling personalised medical support for chronic disease management through a hybrid robot-cloud approach","volume":"41","author":"Fiorini","year":"2017","journal-title":"Auton. Robot."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"5171","DOI":"10.1007\/s11042-016-3553-5","article-title":"Interactive multimedia content for older adults: The case of SeniorChannel","volume":"76","author":"Orso","year":"2017","journal-title":"Multimed. Tools Appl."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1109\/JBHI.2016.2559787","article-title":"Moving real exergaming engines on the web: The webFitForAll case study in an active and healthy ageing living lab environment","volume":"21","author":"Konstantinidis","year":"2017","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"549","DOI":"10.1109\/JBHI.2016.2525725","article-title":"An open architecture to support social and health services in a smart TV environment","volume":"21","author":"Costa","year":"2017","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Pedroli, E., Greci, L., Colombo, D., Serino, S., Cipresso, P., Arlati, S., Mondellini, M., Boilini, L., Giussani, V., and Goulene, K. (2018). Characteristics, usability, and users experience of a system combining cognitive and physical therapy in a virtual environment: Positive bike. Sensors, 18.","DOI":"10.3390\/s18072343"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"1167","DOI":"10.1007\/s12008-018-0467-7","article-title":"Design impact of acceptability and dependability in assisted living robotic applications","volume":"12","author":"Cavallo","year":"2018","journal-title":"Int. J. Interact. Des. Manuf."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1007\/s11370-017-0237-6","article-title":"The multi-modal interface of Robot-Era multi-robot services tailored for the elderly","volume":"11","author":"Broz","year":"2018","journal-title":"Intell. Serv. Robot."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1089\/tmj.2016.0258","article-title":"ROBIN, a telepresence robot to support older users monitoring and social inclusion: Development and evaluation","volume":"24","author":"Cortellessa","year":"2018","journal-title":"Telemed. E-Health"},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"2361","DOI":"10.3906\/elk-1806-124","article-title":"An ambient assisted living system for dementia patients","volume":"27","author":"Yilmaz","year":"2019","journal-title":"Turk. J. Electr. Eng. Comput. Sci."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Chartomatsidis, M., and Goumopoulos, C. (2019, January 2\u20134). A Balance Training Game Tool for Seniors using Microsoft Kinect and 3D Worlds. Proceedings of the 5th International Conference on Information and Communication Technologies for Ageing Well and e-Health, Heraklion, Greece.","DOI":"10.5220\/0007759001350145"},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"732","DOI":"10.1016\/j.procs.2019.12.242","article-title":"Digi&Mind: Development and validation of a multi-domain digital cognitive stimulation program for older adults with cognitive decline","volume":"164","author":"Couto","year":"2019","journal-title":"Procedia Comput. Sci."},{"key":"ref_63","doi-asserted-by":"crossref","unstructured":"Palestra, G., Rebiai, M., Courtial, E., and Koutsouris, D. (2019). Evaluation of a rehabilitation system for the elderly in a day care center. Information, 10.","DOI":"10.3390\/info10010003"},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"1019","DOI":"10.3389\/fphys.2019.01019","article-title":"Exergame-Driven high-Intensity interval training in untrained community dwelling older adults: A Formative one group quasi-experimental feasibility trial","volume":"10","author":"Rebsamen","year":"2019","journal-title":"Front. Physiol."},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Delmastro, F., Dolciotti, C., La Rosa, D., Di Martino, F., Magrini, M., Coscetti, S., and Palumbo, F. (2019). Experimenting Mobile and e-Health Services with Frail MCI Older People. Information, 10.","DOI":"10.3390\/info10080253"},{"key":"ref_66","doi-asserted-by":"crossref","unstructured":"Money, A.G., Atwal, A., Boyce, E., Gaber, S., Windeatt, S., and Alexandrou, K. (2019). Falls Sensei: A serious 3D exploration game to enable the detection of extrinsic home fall hazards for older adults. BMC Med. Inform. Decis. Mak., 19.","DOI":"10.1186\/s12911-019-0808-x"},{"key":"ref_67","doi-asserted-by":"crossref","first-page":"2143","DOI":"10.1007\/s12652-018-0827-y","article-title":"MobileCogniTracker","volume":"10","author":"Hermens","year":"2019","journal-title":"J. Ambient Intell. Humaniz. Comput."},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"304","DOI":"10.3389\/fnagi.2019.00304","article-title":"A pilot study of an in-home multicomponent exergame training for older adults: Feasibility, usability and pre-post evaluation","volume":"11","author":"Adcock","year":"2019","journal-title":"Front. Aging Neurosci."},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"597","DOI":"10.1007\/s12008-018-00522-x","article-title":"Prototyping adaptive systems in smart environments using virtual reality","volume":"13","author":"Menghi","year":"2019","journal-title":"Int. J. Interact. Des. Manuf."},{"key":"ref_70","doi-asserted-by":"crossref","first-page":"268","DOI":"10.1109\/JBHI.2019.2894552","article-title":"Design and usability assessment of a multi-device SOA-based telecare framework for the elderly","volume":"24","author":"Macis","year":"2019","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_71","first-page":"126","article-title":"I-Support: A robotic platform of an assistive bathing robot for the elderly population","volume":"103451","author":"Zlatintsi","year":"2020","journal-title":"Robot. Auton. Syst."},{"key":"ref_72","doi-asserted-by":"crossref","unstructured":"Zhunio, C.S., Orellana, P.C., and Pati\u00f1o, A.V. (2020, January 20\u201322). A Memory Game for Elderly People: Development and Evaluation. Proceedings of the Seventh International Conference on eDemocracy & eGovernment (ICEDEG), Buenos Aires, Argentina.","DOI":"10.1109\/ICEDEG48599.2020.9096862"},{"key":"ref_73","doi-asserted-by":"crossref","first-page":"e14182","DOI":"10.2196\/14182","article-title":"Serious motion-based exercise games for older adults: Evaluation of usability, performance, and pain mitigation","volume":"8","author":"Brauner","year":"2020","journal-title":"JMIR Serious Games"},{"key":"ref_74","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1186\/s11556-019-0233-2","article-title":"A usability study of a multicomponent video game-based training for older adults","volume":"17","author":"Adcock","year":"2020","journal-title":"Eur. Rev. Aging Phys. Act."},{"key":"ref_75","unstructured":"European Union (2007). Opinion of the European Economic and Social Committee on the Proposal for a Decision of the European Parliament and of the Council on the Participation by the Community in a Research and Development Programme AiMed. at Enhancing the Quality of life of Older People through the Use of New Information and Communication Technologies (ICT), Undertaken by Several Member States, European Commission."},{"key":"ref_76","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1016\/j.patrec.2013.07.003","article-title":"Multimodal interaction: A review","volume":"36","author":"Turk","year":"2014","journal-title":"Pattern Recognit. Lett."},{"key":"ref_77","doi-asserted-by":"crossref","first-page":"e15337","DOI":"10.2196\/15337","article-title":"Usability evaluations of mobile mental health technologies: Systematic review","volume":"22","author":"Inal","year":"2020","journal-title":"J. Med. Internet Res."},{"key":"ref_78","doi-asserted-by":"crossref","unstructured":"Almeida, A.F., Rocha, N.P., and Silva, A.G. (2020). Methodological Quality of Manuscripts Reporting on the Usability of Mobile Applications for Pain Assessment and Management: A Systematic Review. Int. J. Environ. Res. Public Health, 17.","DOI":"10.3390\/ijerph17030785"},{"key":"ref_79","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1007\/s10209-013-0328-x","article-title":"Usability, accessibility and ambient-assisted living: A systematic literature review","volume":"14","author":"Silva","year":"2015","journal-title":"Univers. Access Inf. Soc."},{"key":"ref_80","doi-asserted-by":"crossref","first-page":"e22774","DOI":"10.2196\/22774","article-title":"Procedures of User-Centered Usability Assessment for Digital Solutions: Scoping Review of Reviews Reporting on Digital Solutions Relevant for Older Adults","volume":"8","author":"Silva","year":"2021","journal-title":"JMIR Hum. Factors"},{"key":"ref_81","unstructured":"European Commission (2017). Regulation (EU) 2017\/745 of the European Parliament and the Council of 5 April 2017 on Medical Devices, Amending Directive 2001\/83\/EC, Regulation (EC) No 178\/2002 and Regulation (EC) No 1223\/2009 and Repealing Council Directives 90\/385\/EEC and 93\/42\/EEC, European Commission."},{"key":"ref_82","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1177\/1460458216683535","article-title":"The European Union\u2019s Ambient and Assisted Living JoInt. Programme: An evaluation of its impact on population health and well-being","volume":"25","year":"2019","journal-title":"Health Inform. J."},{"key":"ref_83","unstructured":"Lusoli, W. (2020). Reproducibility of Scientific Results in the EU: Scoping Report, Publications Office of the European Union."}],"container-title":["International Journal of Environmental Research and Public Health"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1660-4601\/18\/21\/11507\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:24:26Z","timestamp":1760167466000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1660-4601\/18\/21\/11507"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,1]]},"references-count":83,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2021,11]]}},"alternative-id":["ijerph182111507"],"URL":"https:\/\/doi.org\/10.3390\/ijerph182111507","relation":{},"ISSN":["1660-4601"],"issn-type":[{"type":"electronic","value":"1660-4601"}],"subject":[],"published":{"date-parts":[[2021,11,1]]}}}