{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,21]],"date-time":"2026-02-21T18:32:48Z","timestamp":1771698768085,"version":"3.50.1"},"reference-count":31,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:00:00Z","timestamp":1750291200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Virtual Worlds"],"abstract":"<jats:p>This paper evaluates a new interaction mode for object manipulation tasks in virtual reality (VR) utilizing an aircraft cockpit simulation. Building on prior research, this study examines the effectiveness and user experience of a mixed-interaction mode that involves the combination of handheld controllers with hand gestures. Qualitative interviews with participants provided detailed feedback on the combined input approach. The analysis highlights the strengths and challenges of the mixed-interaction mode, indicating a perceived increase in task completion efficacy and enhanced user experience. As an outcome of the research, design guidelines were developed based on participants\u2019 insights, focusing on the optimal balance of naturalness and precision for mixed interaction in VR that can also be utilized more generally. This study offers practical implications for creating immersive virtual environments and informs future research in VR interaction modes and user experience.<\/jats:p>","DOI":"10.3390\/virtualworlds4020028","type":"journal-article","created":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T06:11:07Z","timestamp":1750313467000},"page":"28","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Exploring Mixed-Interaction Mode in a Virtual Cockpit: Controller and Hand Gesture Integration"],"prefix":"10.3390","volume":"4","author":[{"given":"Yemon","family":"Lee","sequence":"first","affiliation":[{"name":"School of Future Environments, Auckland University of Technology, Private Bag 92006, Auckland 1142, New Zealand"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4046-5959","authenticated-orcid":false,"given":"Andy M.","family":"Connor","sequence":"additional","affiliation":[{"name":"School of Future Environments, Auckland University of Technology, Private Bag 92006, Auckland 1142, New Zealand"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5997-3136","authenticated-orcid":false,"given":"Stefan","family":"Marks","sequence":"additional","affiliation":[{"name":"School of Future Environments, Auckland University of Technology, Private Bag 92006, Auckland 1142, New Zealand"}]}],"member":"1968","published-online":{"date-parts":[[2025,6,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1007\/s12193-024-00431-2","article-title":"Mixed Interaction: Evaluating User Interactions for Object Manipulations in Virtual Space","volume":"18","author":"Lee","year":"2024","journal-title":"J Multimodal User Interfaces"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Hoang, T., Greuter, S., and Taylor, S. (2022, January 12\u201316). An evaluation of virtual reality maintenance training for industrial hydraulic machines. Proceedings of the 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Christchurch, New Zealand.","DOI":"10.1109\/VR51125.2022.00077"},{"key":"ref_3","unstructured":"Dai, F. (2012). Virtual Reality for Industrial Applications, Springer Science & Business Media."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40327-015-0031-5","article-title":"Low-cost virtual reality environment for engineering and construction","volume":"4","author":"Hilfert","year":"2016","journal-title":"Vis. Eng."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Khundam, C. (2015, January 22\u201324). First person movement control with palm normal and hand gesture interaction in virtual reality. Proceedings of the 2015 12th International Joint Conference on Computer Science and Software Engineering (JCSSE), Songkhla, Thailand.","DOI":"10.1109\/JCSSE.2015.7219818"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1007\/s10055-020-00461-7","article-title":"Hand-adaptive user interface: Improved gestural interaction in virtual reality","volume":"25","author":"Lou","year":"2021","journal-title":"Virtual Real."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"303","DOI":"10.3724\/SP.J.2096-5796.2019.0013","article-title":"Influence of Multi-Modality on Moving Target Selection in Virtual Reality","volume":"1","author":"Yang","year":"2019","journal-title":"Virtual Real. Intell. Hardw."},{"key":"ref_8","unstructured":"Perret, J., and Vander Poorten, E. (2018, January 25\u201327). Touching virtual reality: A review of haptic gloves. Proceedings of the 16th International Conference on New Actuators (ACTUATOR 2018), Bremen, Germany."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Rahman, Y., Asish, S.M., Fisher, N.P., Bruce, E.C., Kulshreshth, A.K., and Borst, C.W. (2020, January 22\u201326). Exploring eye gaze visualization techniques for identifying distracted students in educational VR. Proceedings of the 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Christchurch, New Zealand.","DOI":"10.1109\/VR46266.2020.00009"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1481","DOI":"10.1007\/s10055-022-00738-z","article-title":"Eye tracking in virtual reality: A broad review of applications and challenges","volume":"27","author":"Adhanom","year":"2023","journal-title":"Virtual Real."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3491207","article-title":"The eye in extended reality: A survey on gaze interaction and eye tracking in head-worn extended reality","volume":"55","author":"Plopski","year":"2022","journal-title":"ACM Comput. Surv."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"810","DOI":"10.1007\/s11528-022-00717-w","article-title":"A systematic review of research on high-immersion virtual reality for language learning","volume":"66","author":"Dhimolea","year":"2022","journal-title":"TechTrends"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1186\/s40561-023-00272-8","article-title":"Investigating the use of virtual reality to improve speaking skills: Insights from students and teachers","volume":"10","author":"Ironsi","year":"2023","journal-title":"Smart Learn. Environ."},{"key":"ref_14","unstructured":"Sund\u00e9n, E., Lundgren, I., and Ynnerman, A. (2017). Hybrid Virtual Reality Touch Table\u2014An Immersive Collaborative Platform for Public Explanatory Use of Cultural Objects and Sites. Eurographics Workshop on Graphics and Cultural Heritage, Eurographics Association."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Wolf, E., Kl\u00fcber, S., Zimmerer, C., Lugrin, J.-L., and Latoschik, M.E. (2019, January 14\u201318). \u201cPaint That Object Yellow\u201d: Multimodal Interaction to Enhance Creativity During Design Tasks in VR. Proceedings of the 2019 International Conference on Multimodal Interaction, Suzhou, China.","DOI":"10.1145\/3340555.3353724"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1016\/j.cag.2019.06.003","article-title":"2D, 3D, or Speech? A Case Study on Which User Interface is Preferable for What Kind of Object Interaction in Immersive Virtual Reality","volume":"82","author":"Hepperle","year":"2019","journal-title":"Comput. Graph."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2513","DOI":"10.1109\/TVCG.2021.3067759","article-title":"Comparing and Combining Virtual Hand and Virtual Ray Pointer Interactions for Data Manipulation in Immersive Analytics","volume":"27","author":"Wagner","year":"2021","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1007\/s12193-015-0176-5","article-title":"Multimodal Interaction with Virtual Worlds XMMVR: eXtensible Language for MultiModal Interaction with Virtual Reality Worlds","volume":"9","author":"Olmedo","year":"2015","journal-title":"J. Multimodal User Interfaces"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1080\/10447318.2020.1809248","article-title":"Evaluation of a Hybrid of Hand Gesture and Controller Inputs in Virtual Reality","volume":"37","author":"Huang","year":"2021","journal-title":"Int. J. Hum.\u2013Comput. Interact."},{"key":"ref_20","unstructured":"Ionescu, D., Ionescu, B., Gadea, C., and Islam, S. (August, January 31). A Multimodal Interaction Method that Combines Gestures and Physical Game Controllers. Proceedings of the 20th International Conference on Computer Communications and Networks (ICCCN), Lahaina, HI, USA."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10055-016-0293-9","article-title":"Industry Use of Virtual Reality in Product Design and Manufacturing: A Survey","volume":"21","author":"Berg","year":"2017","journal-title":"Virtual Real."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"379","DOI":"10.3758\/BF03195514","article-title":"Beyond the five-user assumption: Benefits of increased sample sizes in usability testing","volume":"35","author":"Faulkner","year":"2003","journal-title":"Behav. Res. Methods Instrum. Comput."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1145\/1735223.1735255","article-title":"Number of people required for usability evaluation: The 10 \u00b1 2 rule","volume":"53","author":"Hwang","year":"2010","journal-title":"Commun. ACM"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1145\/2133806.2133824","article-title":"Sample size in usability studies","volume":"55","author":"Schmettow","year":"2012","journal-title":"Commun. ACM"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1191\/1478088706qp063oa","article-title":"Using Thematic Analysis in Psychology","volume":"3","author":"Braun","year":"2006","journal-title":"Qual. Res. Psychol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1391","DOI":"10.1007\/s11135-021-01182-y","article-title":"A worked example of Braun and Clarke\u2019s approach to reflexive thematic analysis","volume":"56","author":"Byrne","year":"2022","journal-title":"Qual. Quant."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"733","DOI":"10.1002\/nop2.275","article-title":"Qualitative Thematic Analysis Based on Descriptive Phenomenology","volume":"6","author":"Sundler","year":"2019","journal-title":"Nurs. Open"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Cooper, H., Camic, P.M., Long, D.L., Panter, A.T., Rindskopf, D., and Sher, K.J. (2012). Thematic analysis. APA Handbook of Research Methods in Psychology: Vol. 2. Research Designs, American Psychological Association.","DOI":"10.1037\/13620-000"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1609406917733847","DOI":"10.1177\/1609406917733847","article-title":"Thematic Analysis: Striving to Meet the Trustworthiness Criteria","volume":"16","author":"Nowell","year":"2017","journal-title":"Int. J. Qual. Methods"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Kang, H.J., Shin, J., and Ponto, K. (2020, January 22\u201326). A Comparative Analysis of 3D User Interaction: How to Move Virtual Objects in Mixed Reality. Proceedings of the 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Atlanta, GA, USA.","DOI":"10.1109\/VR46266.2020.00047"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"377","DOI":"10.1162\/PRES_a_00207","article-title":"Design Choices and Their Implications for 3D Mid-Air Manipulation Techniques","volume":"23","author":"Bossavit","year":"2014","journal-title":"Presence Teleoperators Virtual Environ."}],"container-title":["Virtual Worlds"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2813-2084\/4\/2\/28\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:54:51Z","timestamp":1760032491000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2813-2084\/4\/2\/28"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,19]]},"references-count":31,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["virtualworlds4020028"],"URL":"https:\/\/doi.org\/10.3390\/virtualworlds4020028","relation":{},"ISSN":["2813-2084"],"issn-type":[{"value":"2813-2084","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,19]]}}}