{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T03:28:03Z","timestamp":1776396483947,"version":"3.51.2"},"reference-count":39,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2021,10,29]],"date-time":"2021-10-29T00:00:00Z","timestamp":1635465600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100010418","name":"Institute for Information and Communications Technology Promotion","doi-asserted-by":"publisher","award":["IITP-2021-2018-0-01431"],"award-info":[{"award-number":["IITP-2021-2018-0-01431"]}],"id":[{"id":"10.13039\/501100010418","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010418","name":"Institute for Information and Communications Technology Promotion","doi-asserted-by":"publisher","award":["2019-0-01347"],"award-info":[{"award-number":["2019-0-01347"]}],"id":[{"id":"10.13039\/501100010418","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100010418","name":"Institute for Information and Communications Technology Promotion","doi-asserted-by":"publisher","award":["2020-0-01373"],"award-info":[{"award-number":["2020-0-01373"]}],"id":[{"id":"10.13039\/501100010418","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>A virtual reality (VR) controller plays a key role in supporting interactions between users and the virtual environment. This paper investigates the relationship between the user experience and VR control device modality. We developed a VR firefighting training system integrated with four control devices adapted from real firefighting tools. We iteratively improved the controllers and VR system through a pilot study with six participants and conducted a user study with 30 participants to assess two salient human factor constructs\u2014perceived presence and cognitive load\u2014with three device modality conditions (two standard VR controllers, four real tools, and a hybrid of one real tool and one standard VR controller). We found that having more realistic devices that simulate real tools does not necessarily guarantee a higher level of user experience, highlighting a strategic approach to the development and utilization of VR control devices. Our study gives empirical insights on establishing appropriate combinations of VR control device modality in the context of field-based VR simulation and training.<\/jats:p>","DOI":"10.3390\/s21217193","type":"journal-article","created":{"date-parts":[[2021,11,1]],"date-time":"2021-11-01T22:24:22Z","timestamp":1635805462000},"page":"7193","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["The More, the Better? Improving VR Firefighting Training System with Realistic Firefighter Tools as Controllers"],"prefix":"10.3390","volume":"21","author":[{"given":"Seunggon","family":"Jeon","sequence":"first","affiliation":[{"name":"Department of Artificial Intelligence, Ajou University, Suwon 16499, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Seungwon","family":"Paik","sequence":"additional","affiliation":[{"name":"Department of Artificial Intelligence, Ajou University, Suwon 16499, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ungyeon","family":"Yang","sequence":"additional","affiliation":[{"name":"Electronics and Telecommunications Research Institute, Daejeon 34129, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2460-0468","authenticated-orcid":false,"given":"Patrick C.","family":"Shih","sequence":"additional","affiliation":[{"name":"School of Informatics, Computing, and Engineering, Indiana University Bloomington, Bloomington, IL 47405, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5535-0081","authenticated-orcid":false,"given":"Kyungsik","family":"Han","sequence":"additional","affiliation":[{"name":"Department of Intelligence Computing, Hanyang University, Seoul 04763, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,10,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Panerai, S., Gelardi, D., Catania, V., Rundo, F., Tasca, D., Musso, S., Prestianni, G., Muratore, S., Babiloni, C., and Ferri, R. (2021). Functional Living Skills: A Non-Immersive Virtual Reality Training for Individuals with Major Neurocognitive Disorders. Sensors, 21.","DOI":"10.3390\/s21175751"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Jeon, S.G., Han, J., Jo, Y., and Han, K. (2019, January 12\u201315). Being more focused and engaged in firefighting training: Applying user-centered design to VR system development. Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology, Parramatta, Australia.","DOI":"10.1145\/3359996.3364268"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"778","DOI":"10.1080\/10494820.2013.815221","article-title":"Evaluating virtual reality and augmented reality training for industrial maintenance and assembly tasks","volume":"23","author":"Gavish","year":"2015","journal-title":"Interact. Learn. Environ."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Khan, N., Muhammad, K., Hussain, T., Nasir, M., Munsif, M., Imran, A.S., and Sajjad, M. (2021). An adaptive game-based learning strategy for children road safety education and practice in virtual space. Sensors, 21.","DOI":"10.3390\/s21113661"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"101","DOI":"10.3389\/frobt.2019.00101","article-title":"A SWOT analysis of the field of virtual reality for firefighter training","volume":"6","author":"Engelbrecht","year":"2019","journal-title":"Front. Robot. AI"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1007\/s10055-020-00447-5","article-title":"Comparing the effectiveness of fire extinguisher virtual reality and video training","volume":"25","author":"Lovreglio","year":"2021","journal-title":"Virtual Real."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"593466","DOI":"10.3389\/fpsyg.2020.593466","article-title":"The Evaluation of Virtual Reality Fire Extinguisher Training","volume":"11","author":"Saghafian","year":"2020","journal-title":"Front. Psychol."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Zenner, A., and Kr\u00fcger, A. (2019, January 4\u20139). Drag on: A virtual reality controller providing haptic feedback based on drag and weight shift. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, UK.","DOI":"10.1145\/3290605.3300441"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Arora, J., Saini, A., Mehra, N., Jain, V., Shrey, S., and Parnami, A. (2019, January 4\u20139). VirtualBricks: Exploring a Scalable, Modular Toolkit for Enabling Physical Manipulation in VR. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, UK.","DOI":"10.1145\/3290605.3300286"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Zhu, K., Chen, T., Han, F., and Wu, Y.S. (2019, January 4\u20139). HapTwist: Creating interactive haptic proxies in virtual reality using low-cost twistable artefacts. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, UK.","DOI":"10.1145\/3290605.3300923"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Shigeyama, J., Hashimoto, T., Yoshida, S., Narumi, T., Tanikawa, T., and Hirose, M. (2019, January 4\u20139). Transcalibur: A weight shifting virtual reality controller for 2d shape rendering based on computational perception model. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, UK.","DOI":"10.1145\/3290605.3300241"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Yi, H., Hong, J., Kim, H., and Lee, W. (2019, January 12\u201315). DexController: Designing a VR Controller with Grasp-Recognition for Enriching Natural Game Experience. Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology, Parramatta, Australia.","DOI":"10.1145\/3359996.3364263"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Seo, S.W., Kwon, S., Hassan, W., Talhan, A., and Jeon, S. (2019, January 12\u201315). Interactive Virtual-Reality Fire Extinguisher with Haptic Feedback. Proceedings of the 25th ACM Symposium on Virtual Reality Software and Technology, Parramatta, Australia.","DOI":"10.1145\/3359996.3364725"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Krell, M. (2015). Evaluating an Instrument to Measure Mental Load and Mental Effort Using Item Response Theory. Cogent Educ., 4.","DOI":"10.1080\/2331186X.2017.1280256"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1111\/j.1467-8535.2004.00373.x","article-title":"Paper-based versus computer-based assessment: Is workload another test mode effect?","volume":"35","author":"Noyes","year":"2004","journal-title":"Br. J. Educ. Technol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1016\/B978-0-12-387691-1.00002-8","article-title":"Cognitive load theory","volume":"Volume 55","author":"Sweller","year":"2011","journal-title":"Psychology of Learning and Motivation"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Jung, S., Wood, A.L., Hoermann, S., Abhayawardhana, P.L., and Lindeman, R.W. (2020, January 22\u201326). The Impact of Multi-sensory Stimuli on Confidence Levels for Perceptual-cognitive Tasks in VR. Proceedings of the 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Atlanta, GA, USA.","DOI":"10.1109\/VR46266.2020.00067"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1016\/j.compedu.2019.01.014","article-title":"Development of fire safety behavioral skills via virtual reality","volume":"133","year":"2019","journal-title":"Comput. Educ."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Clifford, R.M., Jung, S., Hoerrnann, S., Billinqhurst, M., and Lindeman, R.W. (2019, January 23\u201327). Creating a stressful decision making environment for aerial firefighter training in virtual reality. Proceedings of the 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Osaka, Japan.","DOI":"10.1109\/VR.2019.8797889"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Nahavandi, S., Wei, L., Mullins, J., Fielding, M., Deshpande, S., Watson, M., Korany, S., Nahavandi, D., Hettiarachchi, I., and Najdovski, Z. (2019). Haptically-enabled vr-based immersive fire fighting training simulator. Intelligent Computing-Proceedings of the Computing Conference, Springer.","DOI":"10.1007\/978-3-030-22871-2_2"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1162\/pres.1997.6.1.73","article-title":"The effectiveness of virtual reality for administering spatial navigation training to firefighters","volume":"6","author":"Bliss","year":"1997","journal-title":"Presence"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Krekhov, A., Emmerich, K., Bergmann, P., Cmentowski, S., and Kr\u00fcger, J. (2017, January 15\u201318). Self-transforming controllers for virtual reality first person shooters. Proceedings of the Annual Symposium on Computer-Human Interaction in Play, Amsterdam, The Netherlands.","DOI":"10.1145\/3116595.3116615"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"6227","DOI":"10.1007\/s11042-019-08323-4","article-title":"Virtual reality in training: An experimental study with firefighters","volume":"79","author":"Narciso","year":"2020","journal-title":"Multimed. Tools. Appl."},{"key":"ref_24","first-page":"21","article-title":"Basic issues in the use of virtual environments for mental health applications","volume":"58","author":"Rizzo","year":"1998","journal-title":"Stud. Health Technol. Inform."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"262","DOI":"10.1162\/pres.1992.1.2.262","article-title":"Being there: The subjective experience of presence","volume":"1","author":"Heeter","year":"1992","journal-title":"Presence"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1089\/109493101300117884","article-title":"Research on presence in virtual reality: A survey","volume":"4","author":"Schuemie","year":"2001","journal-title":"Cyberpsychol. Behav."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"266","DOI":"10.1162\/105474601300343603","article-title":"The experience of presence: Factor analytic insights","volume":"10","author":"Schubert","year":"2001","journal-title":"Presence"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1002\/acp.1245","article-title":"Impact of reducing intrinsic cognitive load on learning in a mathematical domain","volume":"20","author":"Ayres","year":"2006","journal-title":"Appl. Cogn. Psychol. J. Appl. Res. Mem. Cogn."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1023\/B:TRUC.0000021809.10236.71","article-title":"Designing instructional examples to reduce intrinsic cognitive load: Molar versus modular presentation of solution procedures","volume":"32","author":"Gerjets","year":"2004","journal-title":"Instr. Sci."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"596","DOI":"10.1177\/0735633119867422","article-title":"Influence of students\u2019 learning style, sense of presence, and cognitive load on learning outcomes in an immersive virtual reality learning environment","volume":"58","author":"Huang","year":"2020","journal-title":"J. Educ. Comput. Res."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1016\/j.compedu.2017.05.011","article-title":"Leaderboards within educational videogames: The impact of difficulty, effort and gameplay","volume":"113","author":"Nebel","year":"2017","journal-title":"Comput. Educ."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Luong, T., Martin, N., Raison, A., Argelaguet, F., Diverrez, J.M., and L\u00e9cuyer, A. (2020, January 9\u201313). Towards Real-Time Recognition of Users Mental Workload Using Integrated Physiological Sensors Into a VR HMD. Proceedings of the 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Porto de Galinhas, Brazil.","DOI":"10.1109\/ISMAR50242.2020.00068"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1109\/TAFFC.2016.2582490","article-title":"Cognitive load measurement in a virtual reality-based driving system for autism intervention","volume":"8","author":"Zhang","year":"2017","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1016\/S0005-7967(03)00139-6","article-title":"Treatment of acrophobia in virtual reality: The role of immersion and presence","volume":"42","author":"Krijn","year":"2004","journal-title":"Behav. Res. Ther."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1186\/s12984-016-0112-2","article-title":"Powered wheelchair simulator development: Implementing combined navigation-reaching tasks with a 3D hand motion controller","volume":"13","author":"Tao","year":"2016","journal-title":"J. Neuroeng. Rehabil."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"399","DOI":"10.1016\/j.chb.2017.11.031","article-title":"Meeting others virtually in a day-to-day setting: Investigating social avoidance and prosocial behavior towards avatars and agents","volume":"80","author":"Felnhofer","year":"2018","journal-title":"Comput. Hum. Behav."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1007\/s10339-018-0897-y","article-title":"The experience of virtual reality: Are individual differences in mental imagery associated with sense of presence?","volume":"20","author":"Iachini","year":"2019","journal-title":"Cogn. Process."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1280256","DOI":"10.1080\/2331186X.2017.1280256","article-title":"Evaluating an instrument to measure mental load and mental effort considering different sources of validity evidence","volume":"4","author":"Krell","year":"2017","journal-title":"Cogent. Educ."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Hollan, J., and Stornetta, S. (1992, January 3\u20137). Beyond being there. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Monterey, CA, USA.","DOI":"10.1145\/142750.142769"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/21\/7193\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:22:37Z","timestamp":1760167357000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/21\/7193"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,29]]},"references-count":39,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2021,11]]}},"alternative-id":["s21217193"],"URL":"https:\/\/doi.org\/10.3390\/s21217193","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,29]]}}}