{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,13]],"date-time":"2025-11-13T12:47:58Z","timestamp":1763038078761,"version":"3.41.0"},"reference-count":96,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2025,3,28]],"date-time":"2025-03-28T00:00:00Z","timestamp":1743120000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,3,28]],"date-time":"2025-03-28T00:00:00Z","timestamp":1743120000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000943","name":"Commonwealth Scientific and Industrial Research Organisation","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100000943","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Virtual Reality"],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Empirical study designs in HCI evolve in response to temporal realities and technological advancements. In this context, virtual reality (VR) shows potential for new empirical research designs, going beyond the (still) quite dominantly lab-based research roots in HCI. Previous work has been conducted to identify the use of VR for gathering non-homogeneous and representative sample populations and for conducting empirical studies in resource-constrained environments. Yet, it is unclear how VR empirical user study designs affect the participants\u2019 behavior and experience, potentially influencing the study results compared to in-situ\/in-lab studies. In this paper, we conducted a gesture elicitation study (GES) in a realistic physical smart room and its digital duplicate in VR. Sixty-six participants\u2019 responses were collected using standardized questionnaires along with between-group gesture agreement analysis. Our comparison shows that the VR study produces a higher number of unique gesture proposals and similar best gestures to the in-person study for 95.4% of the referents, with minimum influence on the gesture proposals. We further discuss the usability, pragmatic and hedonic qualities, presence, task load, and implications of using VR for GESs, and highlight future directions for using VR-based empirical study designs. We found that VR can produce reliable data and improve participant experience with the same task load, making it viable to conduct remote GES and a substitute for conventional lab-based experiments.<\/jats:p>","DOI":"10.1007\/s10055-025-01098-0","type":"journal-article","created":{"date-parts":[[2025,3,30]],"date-time":"2025-03-30T23:09:03Z","timestamp":1743376143000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["VR &amp; in-lab GESs: an analysis of empirical research designs for gesture elicitation in in-lab and virtual reality settings"],"prefix":"10.1007","volume":"29","author":[{"given":"Madhawa","family":"Perera","sequence":"first","affiliation":[]},{"given":"Tom","family":"Gedeon","sequence":"additional","affiliation":[]},{"given":"Matt","family":"Adcock","sequence":"additional","affiliation":[]},{"given":"Armin","family":"Haller","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,3,28]]},"reference":[{"key":"1098_CR1","doi-asserted-by":"crossref","unstructured":"Norman DA, Draper SW (1986) User centered system design: New perspectives on human-computer interaction","DOI":"10.1201\/b15703"},{"key":"1098_CR2","doi-asserted-by":"crossref","unstructured":"Wobbrock JO, Morri, MR, Wilson AD (2009) User-defined gestures for surface computing. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1083\u20131092","DOI":"10.1145\/1518701.1518866"},{"key":"1098_CR3","doi-asserted-by":"crossref","unstructured":"Salovaara A, Oulasvirta A, Jacucci G (2017) Evaluation of prototypes and the problem of possible futures. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems 2064\u20132077","DOI":"10.1145\/3025453.3025658"},{"key":"1098_CR4","doi-asserted-by":"crossref","unstructured":"Bargas-Avila JA, Hornb\u00e6k K (2011) Old wine in new bottles or novel challenges: a critical analysis of empirical studies of user experience. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems 2689\u20132698","DOI":"10.1145\/1978942.1979336"},{"key":"1098_CR5","unstructured":"Koeman L (2020) HCI\/UX Research: What Methods do we use?. https:\/\/lisakoeman.nl\/blog\/hci-ux-research-what-methods-do-we-use\/"},{"key":"1098_CR6","doi-asserted-by":"crossref","unstructured":"Wobbrock JO, Aung HH, Rothrock B, Myers BA (2005) Maximizing the guessability of symbolic input. In: CHI\u201905 Extended Abstracts on Human Factors in Computing Systems 1869\u20131872","DOI":"10.1145\/1056808.1057043"},{"key":"1098_CR7","doi-asserted-by":"crossref","unstructured":"Villarreal-Narvaez S, Vanderdonckt J, Vatavu R-D, Wobbrock JO (2020) A systematic review of gesture elicitation studies: What can we learn from 216 studies? In: Proceedings of the 2020 ACM Designing Interactive Systems Conference 855\u2013872","DOI":"10.1145\/3357236.3395511"},{"key":"1098_CR8","doi-asserted-by":"crossref","unstructured":"Magrofuoco N, Vanderdonckt J (2019) Gelicit: a cloud platform for distributed gesture elicitation studies. Proceedings of the ACM on Human-Computer Interaction 3(EICS), 1\u201341","DOI":"10.1145\/3331148"},{"issue":"5","key":"1098_CR9","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3636458","volume":"56","author":"S Villarreal-Narvaez","year":"2024","unstructured":"Villarreal-Narvaez S, Slu\u00ffters A, Vanderdonckt J, Vatavu R-D (2024) Brave new GES world: a systematic literature review of gestures and referents in gesture elicitation studies. ACM Comput Surv 56(5):1\u201355","journal-title":"ACM Comput Surv"},{"key":"1098_CR10","unstructured":"Morris MR, Wobbrock JO, Wilson AD (2010) Understanding users\u2019 preferences for surface gestures. In: Proceedings of Graphics Interface 2010. 261\u2013268"},{"key":"1098_CR11","doi-asserted-by":"crossref","unstructured":"Ali AX, Morris MR, Wobbrock JO (2019) Crowdlicit: A system for conducting distributed end-user elicitation and identification studies. In: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems 1\u201312","DOI":"10.1145\/3290605.3300485"},{"key":"1098_CR12","doi-asserted-by":"crossref","unstructured":"Voit A, Mayer S, Schwind V, Henze N (2019) Online, vr, ar, lab, and in-situ: comparison of research methods to evaluate smart artifacts. In: Proceedings of the 2019 Chi Conference on Human Factors in Computing Systems, pp. 1\u201312","DOI":"10.1145\/3290605.3300737"},{"key":"1098_CR13","doi-asserted-by":"crossref","unstructured":"Mottelson A, Hornb\u00e6k K (2017) Virtual reality studies outside the laboratory. In: Proceedings of the 23rd Acm Symposium on Virtual Reality Software and Technology 1\u201310","DOI":"10.1145\/3139131.3139141"},{"issue":"4","key":"1098_CR14","doi-asserted-by":"publisher","first-page":"486","DOI":"10.1080\/09658211.2021.1904996","volume":"29","author":"P Kourtesis","year":"2021","unstructured":"Kourtesis P, Collina S, Doumas LA, MacPherson SE (2021) An ecologically valid examination of event-based and time-based prospective memory using immersive virtual reality: the effects of delay and task type on everyday prospective memory. Memory 29(4):486\u2013506","journal-title":"Memory"},{"key":"1098_CR15","doi-asserted-by":"crossref","unstructured":"Perera M, Gedeon T, Adcock M, Haller A (2021) Towards self-guided remote user studies-feasibility of gesture elicitation using immersive virtual reality. In: 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2576\u20132583. IEEE","DOI":"10.1109\/SMC52423.2021.9658673"},{"key":"1098_CR16","doi-asserted-by":"crossref","unstructured":"Bellucci A, Zarraonandia T, D\u00edaz P, Aedo I (2021) Welicit: A wizard of oz tool for vr elicitation studies. In: IFIP Conference on Human-Computer Interaction, 82\u201391. Springer","DOI":"10.1007\/978-3-030-85607-6_6"},{"key":"1098_CR17","unstructured":"Henschke M (2020) User behaviour with unguided touchless gestural interfaces. PhD thesis, ANU (Australia)"},{"key":"1098_CR18","doi-asserted-by":"crossref","unstructured":"Nielsen CM, Overgaard M, Pedersen MB, Stage J, Stenild S (2006) It\u2019s worth the hassle! the added value of evaluating the usability of mobile systems in the field. In: Proceedings of the 4th Nordic Conference on Human-computer Interaction: Changing Roles, 272\u2013280","DOI":"10.1145\/1182475.1182504"},{"key":"1098_CR19","doi-asserted-by":"crossref","unstructured":"Rogers Y, Connelly K, Tedesco L, Hazlewood W, Kurtz A, Hall RE, Hursey J, Toscos T (2007) Why it\u2019s worth the hassle: The value of in-situ studies when designing ubicomp. In: International Conference on Ubiquitous Computing, 336\u2013353. Springer","DOI":"10.1007\/978-3-540-74853-3_20"},{"key":"1098_CR20","doi-asserted-by":"crossref","unstructured":"Kjeldskov J, Skov MB, Als BS, H\u00f8egh RT (2004) Is it worth the hassle? exploring the added value of evaluating the usability of context-aware mobile systems in the field. In: International Conference on Mobile Human-Computer Interaction, 61\u201373. Springer","DOI":"10.1007\/978-3-540-28637-0_6"},{"key":"1098_CR21","doi-asserted-by":"crossref","unstructured":"Duh HB-L, Tan GC, Chen VH-h (2006) Usability evaluation for mobile device: a comparison of laboratory and field tests. In: Proceedings of the 8th Conference on Human-computer Interaction with Mobile Devices and Services, 181\u2013186","DOI":"10.1145\/1152215.1152254"},{"key":"1098_CR22","doi-asserted-by":"crossref","unstructured":"Sun X, May A (2013) A comparison of field-based and lab-based experiments to evaluate user experience of personalised mobile devices. Adv Human-Comput Interact 2013","DOI":"10.1155\/2013\/619767"},{"key":"1098_CR23","doi-asserted-by":"crossref","unstructured":"May KR, Gable TM, Walker BN (2017) Designing an in-vehicle air gesture set using elicitation methods. In: Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, 74\u201383","DOI":"10.1145\/3122986.3123015"},{"key":"1098_CR24","doi-asserted-by":"crossref","unstructured":"Fariman HJ, Alyamani HJ, Kavakli M, Hamey L (2016) Designing a user-defined gesture vocabulary for an in-vehicle climate control system. In: Proceedings of the 28th Australian Conference on Computer-Human Interaction, 391\u2013395","DOI":"10.1145\/3010915.3010955"},{"key":"1098_CR25","doi-asserted-by":"crossref","unstructured":"Ruiz J, Li Y, Lank E (2011) User-defined motion gestures for mobile interaction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 197\u2013206","DOI":"10.1145\/1978942.1978971"},{"key":"1098_CR26","doi-asserted-by":"crossref","unstructured":"Kray C, Nesbitt D, Dawson J, Rohs M (2010) User-defined gestures for connecting mobile phones, public displays, and tabletops. In: Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services, 239\u2013248","DOI":"10.1145\/1851600.1851640"},{"key":"1098_CR27","doi-asserted-by":"crossref","unstructured":"Nebeling M, Teunissen E, Husmann M, Norrie MC (2014) Xdkinect: development framework for cross-device interaction using kinect. In: Proceedings of the 2014 ACM SIGCHI Symposium on Engineering Interactive Computing Systems, 65\u201374","DOI":"10.1145\/2607023.2607024"},{"key":"1098_CR28","doi-asserted-by":"crossref","unstructured":"Vatavu R-D, Zaiti I-A (2014) Leap gestures for tv: insights from an elicitation study. In: Proceedings of the ACM International Conference on Interactive Experiences for TV and Online Video, 131\u2013138","DOI":"10.1145\/2602299.2602316"},{"issue":"2","key":"1098_CR29","doi-asserted-by":"publisher","first-page":"733","DOI":"10.1007\/s11042-014-2323-5","volume":"75","author":"H Wu","year":"2016","unstructured":"Wu H, Wang J, Zhang XL (2016) User-centered gesture development in tv viewing environment. Multimedia Tools Appl 75(2):733\u2013760","journal-title":"Multimedia Tools Appl"},{"key":"1098_CR30","doi-asserted-by":"publisher","first-page":"543","DOI":"10.1109\/ACCESS.2015.2432679","volume":"3","author":"H Dong","year":"2015","unstructured":"Dong H, Danesh A, Figueroa N, El Saddik A (2015) An elicitation study on gesture preferences and memorability toward a practical hand-gesture vocabulary for smart televisions. IEEE access 3:543\u2013555","journal-title":"IEEE access"},{"key":"1098_CR31","doi-asserted-by":"crossref","unstructured":"Vogiatzidakis P, Koutsabasis P (2019) Frame-based elicitation of mid-air gestures for a smart home device ecosystem. In: Informatics 6: 23. MDPI","DOI":"10.3390\/informatics6020023"},{"key":"1098_CR32","doi-asserted-by":"crossref","unstructured":"Hoffmann F, Tyroller M-I, Wende F, Henze N (2019) User-defined interaction for smart homes: voice, touch, or mid-air gestures? In: Proceedings of the 18th International Conference on Mobile and Ubiquitous Multimedia, 1\u20137","DOI":"10.1145\/3365610.3365624"},{"issue":"4","key":"1098_CR33","doi-asserted-by":"publisher","first-page":"1196","DOI":"10.1016\/j.apergo.2014.02.010","volume":"45","author":"E Choi","year":"2014","unstructured":"Choi E, Kwon S, Lee D, Lee H, Chung MK (2014) Towards successful user interaction with systems: focusing on user-derived gestures for smart home systems. Appl Ergon 45(4):1196\u20131207","journal-title":"Appl Ergon"},{"key":"1098_CR34","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2021.102755","volume":"159","author":"P Vogiatzidakis","year":"2022","unstructured":"Vogiatzidakis P, Koutsabasis P (2022) \u2018Address and command\u2019: two-handed mid-air interactions with multiple home devices. Int J Human-Comput Stud 159:102755","journal-title":"Int J Human-Comput Stud"},{"key":"1098_CR35","doi-asserted-by":"crossref","unstructured":"Gheran B-F, Vanderdonckt J, Vatavu R-D (2018) Gestures for smart rings: Empirical results, insights, and design implications. In: Proceedings of the 2018 Designing Interactive Systems Conference, 623\u2013635","DOI":"10.1145\/3196709.3196741"},{"issue":"12","key":"1098_CR36","doi-asserted-by":"publisher","first-page":"3479","DOI":"10.1109\/TVCG.2020.3023566","volume":"26","author":"AS Williams","year":"2020","unstructured":"Williams AS, Garcia J, Ortega F (2020) Understanding multimodal user gesture and speech behavior for object manipulation in augmented reality using elicitation. IEEE Trans Visual Comput Gr 26(12):3479\u20133489","journal-title":"IEEE Trans Visual Comput Gr"},{"key":"1098_CR37","doi-asserted-by":"crossref","unstructured":"Piumsomboon T, Clark A, Billinghurst M, Cockburn A (2013) User-defined gestures for augmented reality. In: IFIP Conference on Human-Computer Interaction, 282\u2013299. Springer","DOI":"10.1007\/978-3-642-40480-1_18"},{"key":"1098_CR38","doi-asserted-by":"crossref","unstructured":"Pham T, Vermeulen J, Tang A, MacDonald\u00a0Vermeulen L (2018) Scale impacts elicited gestures for manipulating holograms: Implications for ar gesture design. In: Proceedings of the 2018 Designing Interactive Systems Conference, 227\u2013240","DOI":"10.1145\/3196709.3196719"},{"key":"1098_CR39","doi-asserted-by":"crossref","unstructured":"D\u00f6ring T, Kern D, Marshall P, Pfeiffer M, Sch\u00f6ning J, Gruhn V, Schmidt A (2011) Gestural interaction on the steering wheel: reducing the visual demand. In: Proceedings of the Sigchi Conference on Human Factors in Computing Systems, 483\u2013492","DOI":"10.1145\/1978942.1979010"},{"issue":"2","key":"1098_CR40","doi-asserted-by":"publisher","first-page":"174","DOI":"10.1145\/353485.353487","volume":"7","author":"J Hollan","year":"2000","unstructured":"Hollan J, Hutchins E, Kirsh D (2000) Distributed cognition: toward a new foundation for human-computer interaction research. ACM Trans Comput-Human Interact (TOCHI) 7(2):174\u2013196","journal-title":"ACM Trans Comput-Human Interact (TOCHI)"},{"issue":"11","key":"1098_CR41","doi-asserted-by":"publisher","first-page":"693","DOI":"10.1016\/j.ijhcs.2011.04.005","volume":"69","author":"C K\u00fchnel","year":"2011","unstructured":"K\u00fchnel C, Westermann T, Hemmert F, Kratz S, M\u00fcller A, M\u00f6ller S (2011) I\u2019m home: defining and evaluating a gesture set for smart-home control. Int J Human-Comput Stud 69(11):693\u2013704","journal-title":"Int J Human-Comput Stud"},{"key":"1098_CR42","doi-asserted-by":"crossref","unstructured":"Madapana N, Wachs J (2019) Database of gesture attributes: Zero shot learning for gesture recognition. In: 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), 1\u20138. IEEE","DOI":"10.1109\/FG.2019.8756548"},{"key":"1098_CR43","doi-asserted-by":"crossref","unstructured":"Morris MR, Danielescu A, Drucker S, Fisher D, Lee B, Schraefel M, Wobbrock JO (2014) Reducing legacy bias in gesture elicitation studies. Interactions 21(3)","DOI":"10.1145\/2591689"},{"key":"1098_CR44","doi-asserted-by":"crossref","unstructured":"Chamunorwa M, Wozniak MP, Kr\u00e4mer S, M\u00fcller H, Boll S (2023) An empirical comparison of moderated and unmoderated gesture elicitation studies on soft surfaces and objects for smart home control. Proceedings of the ACM on human-computer interaction 7(MHCI), 1\u201324","DOI":"10.1145\/3604245"},{"issue":"1","key":"1098_CR45","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1145\/291224.291235","volume":"6","author":"B Gaver","year":"1999","unstructured":"Gaver B, Dunne T, Pacenti E (1999) Design: cultural probes. Interactions 6(1):21\u201329","journal-title":"Interactions"},{"key":"1098_CR46","doi-asserted-by":"crossref","unstructured":"Hutchinson H, Mackay W, Westerlund B, Bederson BB, Druin A, Plaisant C, Beaudouin-Lafon M, Conversy S, Evans H, Hansen H (2003) Technology probes: inspiring design for and with families. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 17\u201324","DOI":"10.1145\/642611.642616"},{"key":"#cr-split#-1098_CR47.1","unstructured":"Ratcliffe J, Soave F, Bryan-Kinns N, Tokarchuk L, Farkhatdinov I (2021) Extended reality"},{"key":"#cr-split#-1098_CR47.2","doi-asserted-by":"crossref","unstructured":"(xr) remote research: a survey of drawbacks and opportunities. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 1-13","DOI":"10.1145\/3411764.3445170"},{"key":"1098_CR48","doi-asserted-by":"crossref","unstructured":"Ratcliffe J, Soave F, Hoover M, Ortega FR, Bryan-Kinns N, Tokarchuk L, Farkhatdinov I (2021) Remote xr studies: Exploring three key challenges of remote xr experimentation. In: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, 1\u20134","DOI":"10.1145\/3411763.3442472"},{"key":"1098_CR49","doi-asserted-by":"crossref","unstructured":"Ma X, Cackett M, Park L, Chien E, Naaman M (2018) Web-based vr experiments powered by the crowd. In: Proceedings of the 2018 World Wide Web Conference, 33\u201343","DOI":"10.1145\/3178876.3186034"},{"issue":"4","key":"1098_CR50","doi-asserted-by":"publisher","first-page":"1406","DOI":"10.1109\/TVCG.2016.2518135","volume":"22","author":"A Steed","year":"2016","unstructured":"Steed A, Frlston S, Lopez MM, Drummond J, Pan Y, Swapp D (2016) An \u2018in the wild\u2019 experiment on presence and embodiment using consumer virtual reality equipment. IEEE Trans Visual Comput Gr 22(4):1406\u20131414","journal-title":"IEEE Trans Visual Comput Gr"},{"key":"1098_CR51","doi-asserted-by":"crossref","unstructured":"Silva FV, Mattos Brito\u00a0Oliveira FC, Moraes\u00a0Alves R, Castro\u00a0Quintinho G (2022) Gesture elicitation for augmented reality environments. In: International Conference on Human-Computer Interaction, 143\u2013159. Springer","DOI":"10.1007\/978-3-031-17618-0_12"},{"key":"1098_CR52","doi-asserted-by":"crossref","unstructured":"Perera M, Gedeon T, Haller A, Adcock M (2023) Using virtual reality to overcome legacy bias in remote gesture elicitation studies. In: International Conference on Human-Computer Interaction. Springer","DOI":"10.1007\/978-3-031-35596-7_14"},{"key":"1098_CR53","doi-asserted-by":"crossref","unstructured":"Saffo D, Yildirim C, Di\u00a0Bartolomeo S, Dunne C (2020) Crowdsourcing virtual reality experiments using vrchat. In: Extended Abstracts of the 2020 Chi Conference on Human Factors in Computing Systems, 1\u20138","DOI":"10.1145\/3334480.3382829"},{"issue":"18","key":"1098_CR54","doi-asserted-by":"publisher","first-page":"1747","DOI":"10.1080\/10447318.2019.1572352","volume":"35","author":"P Koutsabasis","year":"2019","unstructured":"Koutsabasis P, Vogiatzidakis P (2019) Empirical research in mid-air interaction: a systematic review. Int J Human-Comput Interact 35(18):1747\u20131768","journal-title":"Int J Human-Comput Interact"},{"issue":"2","key":"1098_CR55","doi-asserted-by":"publisher","first-page":"428","DOI":"10.3758\/BRM.40.2.428","volume":"40","author":"F Dandurand","year":"2008","unstructured":"Dandurand F, Shultz TR, Onishi KH (2008) Comparing online and lab methods in a problem-solving experiment. Behav Res Methods 40(2):428\u2013434","journal-title":"Behav Res Methods"},{"issue":"2","key":"1098_CR56","doi-asserted-by":"publisher","first-page":"120","DOI":"10.1017\/xps.2014.5","volume":"1","author":"S Clifford","year":"2014","unstructured":"Clifford S, Jerit J (2014) Is there a cost to convenience? an experimental comparison of data quality in laboratory and online studies. J Exp Polit Sci 1(2):120\u2013131","journal-title":"J Exp Polit Sci"},{"issue":"5","key":"1098_CR57","doi-asserted-by":"publisher","first-page":"847","DOI":"10.3758\/s13423-012-0296-9","volume":"19","author":"L Germine","year":"2012","unstructured":"Germine L, Nakayama K, Duchaine BC, Chabris CF, Chatterjee G, Wilmer JB (2012) Is the web as good as the lab? comparable performance from web and lab in cognitive\/perceptual experiments. Psychon Bull Rev 19(5):847\u2013857","journal-title":"Psychon Bull Rev"},{"key":"1098_CR58","doi-asserted-by":"crossref","unstructured":"Andreasen MS, Nielsen HV, Schr\u00f8der SO, Stage J (2007) What happened to remote usability testing? an empirical study of three methods. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1405\u20131414","DOI":"10.1145\/1240624.1240838"},{"key":"1098_CR59","unstructured":"Sharp H, Rogers Y, Preece J (2007) Interaction Design: beyond Human-computer Interaction. Wiley"},{"key":"1098_CR60","doi-asserted-by":"crossref","unstructured":"Gustarini M, Ickin S, Wac K (2013) Evaluation of challenges in human subject studies\" in-the-wild\" using subjects\u2019 personal smartphones. In: Proceedings of the 2013 ACM Conference on Pervasive and Ubiquitous Computing Adjunct Publication, 1447\u20131456","DOI":"10.1145\/2494091.2496041"},{"key":"1098_CR61","doi-asserted-by":"crossref","unstructured":"Nielsen M, St\u00f6rring M, Moeslund TB, Granum E (2003) A procedure for developing intuitive and ergonomic gesture interfaces for HCI. In: International Gesture Workshop. Springer","DOI":"10.1007\/978-3-540-24598-8_38"},{"key":"1098_CR62","doi-asserted-by":"crossref","unstructured":"Rovelo\u00a0Ruiz GA, Vanacken D, Luyten K, Abad F, Camahort E (2014) Multi-viewer gesture-based interaction for omni-directional video. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 4077\u20134086","DOI":"10.1145\/2556288.2557113"},{"key":"1098_CR63","doi-asserted-by":"crossref","unstructured":"Troiano GM, Pedersen EW, Hornb\u00e6k K (2014) User-defined gestures for elastic, deformable displays. In: Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces, 1\u20138","DOI":"10.1145\/2598153.2598184"},{"key":"1098_CR64","doi-asserted-by":"crossref","unstructured":"Leng HY, Norowi NM, Jantan AH (2017) A user-defined gesture set for music interaction in immersive virtual environment. In: Proceedings of the 3rd International Conference on Human-Computer Interaction and User Experience in Indonesia, 44\u201351","DOI":"10.1145\/3077343.3077348"},{"key":"1098_CR65","doi-asserted-by":"crossref","unstructured":"Masai K, Kunze K, Sakamoto D, Sugiura Y, Sugimoto M (2020) Face commands-user-defined facial gestures for smart glasses. In: 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 374\u2013386. IEEE","DOI":"10.1109\/ISMAR50242.2020.00064"},{"key":"1098_CR66","doi-asserted-by":"crossref","unstructured":"Bader P, Le HV, Strotzer J, Henze N (2017) Exploring interactions with smart windows for sunlight control. In: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems, 2373\u20132380","DOI":"10.1145\/3027063.3053242"},{"key":"1098_CR67","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.ijhcs.2015.02.010","volume":"80","author":"C Silpasuwanchai","year":"2015","unstructured":"Silpasuwanchai C, Ren X (2015) Designing concurrent full-body gestures for intense gameplay. Int J Human-Comput Stud 80:1\u201313","journal-title":"Int J Human-Comput Stud"},{"key":"1098_CR68","doi-asserted-by":"crossref","unstructured":"Dim NK, Silpasuwanchai C, Sarcar S, Ren X (2016) Designing mid-air tv gestures for blind people using user-and choice-based elicitation approaches. In: Proceedings of the 2016 ACM Conference on Designing Interactive Systems, 204\u2013214","DOI":"10.1145\/2901790.2901834"},{"key":"1098_CR69","doi-asserted-by":"crossref","unstructured":"Cafaro F, Lyons L, Kang R, Radinsky J, Roberts J, Vogt K (2014) Framed guessability: using embodied allegories to increase user agreement on gesture sets. In: Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction, 197\u2013204","DOI":"10.1145\/2540930.2540944"},{"key":"1098_CR70","doi-asserted-by":"crossref","unstructured":"Cafaro F, Lyons L, Antle AN (2018) Framed guessability: Improving the discoverability of gestures and body movements for full-body interaction. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 1\u201312","DOI":"10.1145\/3173574.3174167"},{"key":"1098_CR71","doi-asserted-by":"crossref","unstructured":"Ali A, Ringel\u00a0Morris M, O.\u00a0Wobbrock J, (2021) \u201cI am iron man\u201d priming improves the learnability and memorability of user-elicited gestures. In: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 1\u201314","DOI":"10.1145\/3411764.3445758"},{"key":"1098_CR72","doi-asserted-by":"crossref","unstructured":"Gero KI, Chilton L, Melancon C, Cleron M (2022) Eliciting gestures for novel note-taking interactions. In: Proceedings of the 2022 ACM Designing Interactive Systems Conference, 966\u2013975","DOI":"10.1145\/3532106.3533480"},{"key":"1098_CR73","doi-asserted-by":"crossref","unstructured":"Ch NAN, Tosca D, Crump T, Ansah A, Kun A, Shaer O (2022) Gesture and voice commands to interact with ar windshield display in automated vehicle: a remote elicitation study. In: Proceedings of the 14th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, 171\u2013182","DOI":"10.1145\/3543174.3545257"},{"key":"1098_CR74","doi-asserted-by":"crossref","unstructured":"Lee S-S, Chae J, Kim H, Lim Y-k, Lee K-p (2013) Towards more natural digital content manipulation via user freehand gestural interaction in a living room. In: Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 617\u2013626","DOI":"10.1145\/2493432.2493480"},{"issue":"1","key":"1098_CR75","doi-asserted-by":"publisher","first-page":"395","DOI":"10.1177\/160940691401300121","volume":"13","author":"BK Dennis","year":"2014","unstructured":"Dennis BK (2014) Understanding participant experiences: reflections of a novice research participant. Int J Qualitative Methods 13(1):395\u2013410","journal-title":"Int J Qualitative Methods"},{"key":"1098_CR76","unstructured":"Field A, Hole G (2002) How to Design and Report Experiments. Sage"},{"key":"1098_CR77","unstructured":"Lazar J, Feng JH, Hochheiser H (2017) Research Methods in Human-computer Interaction. Morgan Kaufmann"},{"key":"1098_CR78","doi-asserted-by":"publisher","first-page":"108952","DOI":"10.1109\/ACCESS.2020.2998358","volume":"8","author":"A Fuller","year":"2020","unstructured":"Fuller A, Fan Z, Day C, Barlow C (2020) Digital twin: enabling technologies, challenges and open research. IEEE access 8:108952\u2013108971","journal-title":"IEEE access"},{"key":"1098_CR79","doi-asserted-by":"crossref","unstructured":"Hassenzahl M, Burmester M, Koller F (2003) Attrakdiff: Ein fragebogen zur messung wahrgenommener hedonischer und pragmatischer qualit\u00e4t. In: Mensch & Computer 2003, 187\u2013196. Springer","DOI":"10.1007\/978-3-322-80058-9_19"},{"issue":"194","key":"1098_CR80","first-page":"4","volume":"189","author":"J Brooke","year":"1996","unstructured":"Brooke J (1996) Sus-a quick and dirty usability scale. Usability Eval Ind 189(194):4\u20137","journal-title":"Usability Eval Ind"},{"key":"1098_CR81","doi-asserted-by":"crossref","unstructured":"Hart SG (2006) Nasa-task load index (nasa-tlx); 20 years later. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 50, pp. 904\u2013908. Sage publications Sage CA: Los Angeles, CA","DOI":"10.1177\/154193120605000909"},{"key":"1098_CR82","doi-asserted-by":"crossref","unstructured":"Hart SG, Staveland LE (1988) Development of nasa-tlx (task load index): Results of empirical and theoretical research. In: Advances in Psychology 52: 139\u2013183. Elsevier","DOI":"10.1016\/S0166-4115(08)62386-9"},{"issue":"31","key":"1098_CR83","doi-asserted-by":"publisher","first-page":"12846","DOI":"10.1073\/pnas.1306779110","volume":"110","author":"D Banakou","year":"2013","unstructured":"Banakou D, Groten R, Slater M (2013) Illusory ownership of a virtual child body causes overestimation of object sizes and implicit attitude changes. Proc Natl Acad Sci 110(31):12846\u201312851","journal-title":"Proc Natl Acad Sci"},{"key":"1098_CR84","doi-asserted-by":"publisher","first-page":"141","DOI":"10.3389\/fnhum.2015.00141","volume":"9","author":"K Kilteni","year":"2015","unstructured":"Kilteni K, Maselli A, Kording KP, Slater M (2015) Over my fake body: body ownership illusions for studying the multisensory basis of own-body perception. Front Human Neurosci 9:141","journal-title":"Front Human Neurosci"},{"key":"1098_CR85","doi-asserted-by":"crossref","unstructured":"Kittur A, Chi EH, Suh B (2008) Crowdsourcing user studies with mechanical turk. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 453\u2013456","DOI":"10.1145\/1357054.1357127"},{"key":"1098_CR86","doi-asserted-by":"crossref","unstructured":"Vatavu R-D, Wobbrock JO (2015) Formalizing agreement analysis for elicitation studies: new measures, significance test, and toolkit. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, 1325\u20131334","DOI":"10.1145\/2702123.2702223"},{"key":"1098_CR87","doi-asserted-by":"publisher","first-page":"213","DOI":"10.1007\/s10055-016-0294-8","volume":"20","author":"L Pouliquen-Lardy","year":"2016","unstructured":"Pouliquen-Lardy L, Milleville-Pennel I, Guillaume F, Mars F (2016) Remote collaboration in virtual reality: asymmetrical effects of task distribution on spatial processing and mental workload. Virt Real 20:213\u2013220","journal-title":"Virt Real"},{"key":"1098_CR88","doi-asserted-by":"crossref","unstructured":"Holzwarth V, Schneider J, Handali J, Gisler J, Hirt C, Kunz A, Brocke J (2021) Towards estimating affective states in virtual reality based on behavioral data. Virt Real 1\u201314","DOI":"10.1007\/s10055-021-00518-1"},{"key":"1098_CR89","doi-asserted-by":"crossref","unstructured":"Lu F, Yu D, Liang H-N, Chen W, Papangelis K, Ali NM (2018) Evaluating engagement level and analytical support of interactive visualizations in virtual reality environments. In: 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 143\u2013152. IEEE","DOI":"10.1109\/ISMAR.2018.00050"},{"issue":"3","key":"1098_CR90","doi-asserted-by":"publisher","first-page":"225","DOI":"10.1162\/105474698565686","volume":"7","author":"BG Witmer","year":"1998","unstructured":"Witmer BG, Singer MJ (1998) Measuring presence in virtual environments: a presence questionnaire. Presence 7(3):225\u2013240","journal-title":"Presence"},{"issue":"3","key":"1098_CR91","doi-asserted-by":"publisher","first-page":"76","DOI":"10.1109\/MCG.2009.55","volume":"29","author":"M Slater","year":"2009","unstructured":"Slater M, Khanna P, Mortensen J, Yu I (2009) Visual realism enhances realistic response in an immersive virtual environment. IEEE Comput Gr Appl 29(3):76\u201384","journal-title":"IEEE Comput Gr Appl"},{"key":"1098_CR92","doi-asserted-by":"crossref","unstructured":"Gheran B-F, Villarreal-Narvaez S, Vatavu R-D, Vanderdonckt J (2022) Repliges and gestory: visual tools for systematizing and consolidating knowledge on user-defined gestures. In: Proceedings of the 2022 International Conference on Advanced Visual Interfaces, 1\u20139","DOI":"10.1145\/3531073.3531112"},{"issue":"6","key":"1098_CR93","doi-asserted-by":"publisher","first-page":"0198092","DOI":"10.1371\/journal.pone.0198092","volume":"13","author":"N Madapana","year":"2018","unstructured":"Madapana N, Gonzalez G, Rodgers R, Zhang L, Wachs JP (2018) Gestures for picture archiving and communication systems (pacs) operation in the operating room: Is there any standard? PloS One 13(6):0198092","journal-title":"PloS One"},{"key":"1098_CR94","doi-asserted-by":"crossref","unstructured":"O\u2019hara K, Harper R, Mentis H, Sellen A, Taylor A (2013) On the naturalness of touchless: putting the \u201cinteraction\u201d back into NUI. ACM Trans Comput-Human Interact (TOCHI) 20(1): 1\u201325","DOI":"10.1145\/2442106.2442111"},{"key":"1098_CR95","doi-asserted-by":"crossref","unstructured":"Young G, Milne H, Griffiths D, Padfield E, Blenkinsopp R, Georgiou O (2020) Designing mid-air haptic gesture controlled user interfaces for cars. Proc ACM Human-Comput Interact 4(EICS), 1\u201323","DOI":"10.1145\/3397869"}],"container-title":["Virtual Reality"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10055-025-01098-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10055-025-01098-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10055-025-01098-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,21]],"date-time":"2025-06-21T09:58:42Z","timestamp":1750499922000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10055-025-01098-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,28]]},"references-count":96,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["1098"],"URL":"https:\/\/doi.org\/10.1007\/s10055-025-01098-0","relation":{},"ISSN":["1434-9957"],"issn-type":[{"type":"electronic","value":"1434-9957"}],"subject":[],"published":{"date-parts":[[2025,3,28]]},"assertion":[{"value":"24 December 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 January 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 March 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"No potential conflict of interest was reported by the authors.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"The study was approved by the ethics committee of the Australian National University (ANU)(ethics protocol 2020\/012) and was conducted according to the ANU ethics guidelines. The ANU Ethics Committee reviewed and approved the study protocol to ensure that it met the ethical standards for research involving human subjects. All the participants were provided with information regarding the aims and procedures of the study, and their informed consent was obtained before the study.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}}],"article-number":"59"}}