{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T08:32:47Z","timestamp":1784104367240,"version":"3.55.0"},"reference-count":44,"publisher":"Wiley","license":[{"start":{"date-parts":[[2021,5,4]],"date-time":"2021-05-04T00:00:00Z","timestamp":1620086400000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"ECCI and CITIC at the University of Costa Rica","award":["834-B4-412"],"award-info":[{"award-number":["834-B4-412"]}]},{"name":"ECCI and CITIC at the University of Costa Rica","award":["326-B9-107"],"award-info":[{"award-number":["326-B9-107"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Advances in Human-Computer Interaction"],"published-print":{"date-parts":[[2021,5,4]]},"abstract":"<jats:p>Background. Standardized questionnaires are well-known, reliable, and inexpensive instruments to evaluate user experience (UX). Although the structure, content, and application procedure of the three most recognized questionnaires (AttrakDiff, UEQ, and meCUE) are known, there is no systematic literature review (SLR) that classifies how these questionnaires have been used in primary studies reported academically. This SLR seeks to answer five research questions (RQs), starting with identifying the uses of each questionnaire over the years and by geographic region (RQ1) and the median number of participants per study (how many participants is considered enough when evaluating UX?) (RQ2). This work also aims to establish whether these questionnaires are combined with other evaluation instruments and with which complementary instruments are they used more frequently (RQ3). In addition, this review intends to determine how the three questionnaires have been applied in the fields of ubiquitous computing and ambient intelligence (RQ4) and also in studies that incorporate nontraditional interfaces, such as haptic, gesture, or speech interfaces, to name a few (RQ5). Methods. A systematic literature review was conducted starting from 946 studies retrieved from four digital databases. The main inclusion criteria being the study describes a primary study reported academically, where the standardized questionnaire is used as a UX evaluation instrument in its original and complete form. In the first phase, 189 studies were discarded by screening the title, abstract, and keyword list. In the second phase, 757 studies were full-text reviewed, and 209 were discarded due to the inclusion\/exclusion criteria. The 548 resulting studies were analyzed in detail. Results. AttrakDiff is the questionnaire that counts the most uses since 2006, when the first studies appeared. However, since 2017, UEQ has far surpassed AttrakDiff in uses per year. The contribution of meCUE is still minimal. Europe is the region with the most extended use, followed by Asia. Within Europe, Germany greatly exceeds the rest of countries (RQ1). The median number of participants per study is 20, considering the aggregated data from the three questionnaires. However, this median rises to 30 participants in journal studies while it stays in 20 in conference studies (RQ2). Almost 4 in 10 studies apply the questionnaire as the only evaluation instrument. The remaining studies used between one and five complementary instruments, among which the System Usability Scale (SUS) stands out (RQ3). About 1 in 4 studies analyzed belong to ubiquitous computing and ambient intelligence fields, in which UEQ increases the percentage of uses when compared to its general percentage, particularly in topics such as IoT and wearable interfaces. However, AttrakDiff remains the predominant questionnaire for studies in smart cities and homes and in-vehicle information systems (RQ4). Around 1 in 3 studies include nontraditional interfaces, being virtual reality and gesture interfaces the most numerous. Percentages of UEQ and meCUE uses in these studies are higher than their respective global percentages, particularly in studies using virtual reality and eye tracking interfaces. AttrakDiff maintains its overall percentage in studies with tangible and gesture interfaces and exceeds it in studies with nontraditional visual interfaces, such as displays in windshields or motorcycle helmets (RQ5).<\/jats:p>","DOI":"10.1155\/2021\/5518722","type":"journal-article","created":{"date-parts":[[2021,5,5]],"date-time":"2021-05-05T01:50:09Z","timestamp":1620179409000},"page":"1-22","source":"Crossref","is-referenced-by-count":39,"title":["UX Evaluation with Standardized Questionnaires in Ubiquitous Computing and Ambient Intelligence: A Systematic Literature Review"],"prefix":"10.1155","volume":"2021","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4760-8705","authenticated-orcid":true,"given":"Ignacio","family":"D\u00edaz-Oreiro","sequence":"first","affiliation":[{"name":"Information and Communication Technologies Research Center, University of Costa Rica, San Jos\u00e9 11501-2060, Costa Rica"},{"name":"School of Computer Science and Informatics, University of Costa Rica, San Jos\u00e9 11501-2060, Costa Rica"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0510-8857","authenticated-orcid":true,"given":"Gustavo","family":"L\u00f3pez","sequence":"additional","affiliation":[{"name":"Information and Communication Technologies Research Center, University of Costa Rica, San Jos\u00e9 11501-2060, Costa Rica"},{"name":"School of Computer Science and Informatics, University of Costa Rica, San Jos\u00e9 11501-2060, Costa Rica"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5612-0699","authenticated-orcid":true,"given":"Luis","family":"Quesada","sequence":"additional","affiliation":[{"name":"School of Computer Science and Informatics, University of Costa Rica, San Jos\u00e9 11501-2060, Costa Rica"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3989-979X","authenticated-orcid":true,"given":"Luis A.","family":"Guerrero","sequence":"additional","affiliation":[{"name":"School of Computer Science and Informatics, University of Costa Rica, San Jos\u00e9 11501-2060, Costa Rica"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","reference":[{"key":"1","doi-asserted-by":"crossref","article-title":"Applying the user experience questionnaire (UEQ) in different evaluation scenarios","author":"M. Schrepp","DOI":"10.1007\/978-3-319-07668-3_37"},{"key":"2","first-page":"521","article-title":"User experience evaluation methods","author":"A. P. O. S. Vermeeren"},{"key":"3","doi-asserted-by":"crossref","article-title":"The UX metrics table: a missing artifact","author":"D. Wallach","DOI":"10.1007\/978-3-319-58634-2_37"},{"key":"4","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/j.chb.2014.10.048","article-title":"User experience: a concept without consensus? Exploring practi-tioners\u2019 perspectives through an international survey","volume":"43","author":"C. Lallemand","year":"2015","journal-title":"Computers in Human Behavior"},{"key":"5","article-title":"User experience white paper: bringing clarity to the concept of user experience","author":"V. Roto"},{"key":"6","article-title":"Towards a shared definition of user experience","author":"E. Law"},{"key":"7","volume-title":"ISO DIS 9241-210:2010 Ergonomics of Human-System Interaction","author":"ISO","year":"2010"},{"key":"8","article-title":"When to use which user-experience research methods","author":"C. Rohrer","year":"2014"},{"key":"9","volume-title":"M\u00e9thodes de design UX. 30 m\u00e9thodes fondamentales pour concevoir des exp\u00e9riences optimales","author":"C. Lallemand","year":"2018","edition":"2nd"},{"key":"10","volume-title":"Mesurer la formation de la premi\u00e8re impression d\u2019une interface \u00e0 l\u2019aide du test des 5 secondes; Huiti\u00e8me Colloque de Psychologie Ergonomique (EPIQUE)","author":"G. Gronier","year":"2015"},{"key":"11","doi-asserted-by":"crossref","article-title":"How could an intranet be like a friend to me?","author":"C. Lallemand","DOI":"10.1145\/3121283.3121288"},{"key":"12","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1016\/j.ijhcs.2018.08.008","article-title":"No need to read: developing a pictorial single-item scale for measuring perceived usability","volume":"122","author":"J. Baumgartner","year":"2019","journal-title":"International Journal of Human-Computer Studies"},{"key":"13","first-page":"278","article-title":"How usability can save the day - methodological considerations for making automated driving a success story","author":"Y. Forster"},{"key":"14","volume-title":"Design, User Experience, and Usability: Users, Contexts and Case Studies","author":"J. Klammer","year":"2018"},{"key":"15","volume-title":"A Systematic Review about User Experience Evaluation. In Design, User Experience, and Usability: Design Thinking and Methods","author":"C. L. B. Maia","year":"2016"},{"key":"16","first-page":"492","article-title":"A systematic review of user experience evaluation methods in information driven websites","author":"A. C. Ten"},{"key":"17","volume-title":"Technique for the Measurement of Attitude","author":"R. A. Likert","year":"1932"},{"key":"18","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1037\/h0055737","article-title":"The Nature and measurement of meaning","volume":"49","author":"E. C. Osgood","year":"1952","journal-title":"Psychological Bulletin"},{"key":"19","first-page":"187","article-title":"Ein Fragebogen zur messung wahrgenommener hedonischer und pragmatischer qualit\u00e4t","author":"M. Hassenzahl"},{"key":"20","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-89350-9_6"},{"key":"21","first-page":"89","article-title":"meCUE \u2013 ein modularer fragebogen zur erfassung des nutzungserlebens [meCUE - a modular questionnaire for capturing the user experience]","volume":"9","author":"M. Minge","year":"2013","journal-title":"Mensch und Computer"},{"key":"22","doi-asserted-by":"crossref","first-page":"481","DOI":"10.1207\/S15327590IJHC1304_07","article-title":"The effect of perceived hedonic quality on product appealingness","volume":"13","author":"M. Hassenzahl","year":"2001","journal-title":"International Journal of Human-Computer Interaction"},{"key":"23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/1-4020-2967-5_4","article-title":"The thing and I: understanding the relationship between user and product","volume-title":"Funology :\u2009From Usability to Enjoyment","author":"M. Hassenzahl","year":"2003"},{"issue":"4","key":"24","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1080\/00207590701396674","article-title":"Usability, aesthetics and emotions in human-technologyinteraction","volume":"42","author":"M. Th\u00fcring","year":"2007","journal-title":"International Journal of Psychology"},{"key":"25","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4419-1428-6","volume-title":"Ubiquitous Computing. In Encyclopedia of the Sciences of Learning","author":"M. Seel","year":"2012"},{"issue":"3","key":"26","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1038\/scientificamerican0991-94","article-title":"The computer for the 21 st century","volume":"265","author":"M. Weiser","year":"1991","journal-title":"Scientific American"},{"key":"27","volume-title":"Scenarios for Ambient Intelligence in 2010. IST Advisory Group Final Report","author":"K. Ducatel","year":"2001"},{"key":"28","doi-asserted-by":"crossref","DOI":"10.1007\/0-387-22991-4_1","article-title":"Ambient Intelligence. a gentle introduction","volume-title":"Ambient Intelligence","author":"P. Remagnino","year":"2005"},{"key":"29","doi-asserted-by":"crossref","DOI":"10.1007\/b138670","volume-title":"Ambient Intelligence","author":"W. Weber","year":"2005"},{"key":"30","article-title":"Ambient intelligence: basic concepts and applications","volume-title":"Software and Data Technologies. ICSOFT 2006. Communications in Computer and Information Science","author":"J. C. Augusto","year":"2008"},{"issue":"1","key":"31","doi-asserted-by":"crossref","first-page":"71","DOI":"10.3233\/AIS-180508","article-title":"Artificial intelligence and ambient intelligence","volume":"11","author":"M. Gams","year":"2019","journal-title":"Journal of Ambient Intelligence and Smart Environments"},{"issue":"3","key":"32","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1016\/j.tele.2004.11.001","article-title":"Perspectives of ambient intelligence in the home environment","volume":"22","author":"M. Friedewald","year":"2005","journal-title":"Telematics and Informatics"},{"key":"33","article-title":"Ambient intelligence and ubiquitous computing","volume-title":"Handbook on Information Technologies for Education and Training. International Handbooks on Information Systems","author":"M. Bick","year":"2008"},{"key":"34","doi-asserted-by":"crossref","first-page":"137","DOI":"10.21307\/ijssis-2017-283","article-title":"Human-computer interaction: overview on state of the art","volume":"1","author":"F. Karray","year":"2008","journal-title":"International Journal on Smart Sensing and Intelligent Systems"},{"key":"35","volume-title":"HCI beyond GUI: Design for Haptic, Speech, Olfactory and Other Nontraditional Interfaces","author":"P. Kortum","year":"2008","edition":"1st"},{"key":"36","first-page":"e1","article-title":"The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions","volume":"62","author":"A. Liberati","year":"2009","journal-title":"Explanation and Elaboration"},{"key":"37","first-page":"115","volume-title":"The meCUE Questionnaire: A Modular Tool for Measuring User Experience","author":"M. Minge","year":"2017"},{"key":"38","first-page":"451","article-title":"The MeCUE questionnaire (2.0): meeting five basic requirements for lean and standardized UX assessment","author":"M. Minge"},{"key":"39","doi-asserted-by":"crossref","article-title":"The gods play dice together: the influence of social elements of gamification on seniors\u2019 user experience","author":"I. Wagner","DOI":"10.1007\/978-3-319-21380-4_57"},{"key":"40","first-page":"509","article-title":"Design. Can we distinguish pragmatic from hedonic user experience qualities with implicit measures?","author":"K. Pollmann"},{"key":"41","first-page":"1249","article-title":"Aesthetic categories of interaction: aesthetic perceptions on smartphone and computer","author":"M. M\u00f5ttus"},{"key":"42","doi-asserted-by":"crossref","first-page":"15587","DOI":"10.1109\/ACCESS.2018.2817022","article-title":"Data-Driven knowledge acquisition system: an end-to-end knowledge engineering process for generating production rules","volume":"6","author":"M. A. Ali","year":"2018","journal-title":"IEEE Access"},{"key":"43","first-page":"31","article-title":"An evaluation of a smart-phone-based menu system for immersive virtual environments","author":"S. Gebhardt"},{"key":"44","doi-asserted-by":"crossref","first-page":"644","DOI":"10.1109\/TVCG.2013.31","article-title":"Extended pie menus for immersive virtual environments","volume":"19","author":"S. Gebhardt","year":"2013","journal-title":"IEEE Transactions on Visualization and Computer Graphics"}],"container-title":["Advances in Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/downloads.hindawi.com\/journals\/ahci\/2021\/5518722.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/ahci\/2021\/5518722.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/ahci\/2021\/5518722.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,5,5]],"date-time":"2021-05-05T01:51:07Z","timestamp":1620179467000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.hindawi.com\/journals\/ahci\/2021\/5518722\/"}},"subtitle":[],"editor":[{"given":"Alice","family":"Chirico","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2021,5,4]]},"references-count":44,"alternative-id":["5518722","5518722"],"URL":"https:\/\/doi.org\/10.1155\/2021\/5518722","relation":{},"ISSN":["1687-5907","1687-5893"],"issn-type":[{"value":"1687-5907","type":"electronic"},{"value":"1687-5893","type":"print"}],"subject":[],"published":{"date-parts":[[2021,5,4]]}}}