{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:47:31Z","timestamp":1760240851776,"version":"build-2065373602"},"reference-count":41,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2019,10,3]],"date-time":"2019-10-03T00:00:00Z","timestamp":1570060800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Informatics"],"abstract":"<jats:p>Virtual reality (VR) technologies have opened new possibilities for creating engaging educational games. This paper presents a serious VR game that immerses players into the activities of lunar exploration missions in a virtual environment. We designed and implemented the VR game with the goal of increasing players\u2019 interest in space science. The game motivates players to learn more about historical facts of space missions that astronauts performed on the Moon in the 1970s. We studied usability and engagement of the game through user experience in both VR and non-VR versions of the game. The experimental results show that the VR version improved their engagement and enhanced the interest of players in learning more about the events of lunar exploration.<\/jats:p>","DOI":"10.3390\/informatics6040044","type":"journal-article","created":{"date-parts":[[2019,10,4]],"date-time":"2019-10-04T04:12:52Z","timestamp":1570162372000},"page":"44","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Usability and Engagement Study for a Serious Virtual Reality Game of Lunar Exploration Missions"],"prefix":"10.3390","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0888-8289","authenticated-orcid":false,"given":"Lizhou","family":"Cao","sequence":"first","affiliation":[{"name":"School of Interactive Games and Media, Rochester Institute of Technology, Rochester, NY 14623, USA"}]},{"given":"Chao","family":"Peng","sequence":"additional","affiliation":[{"name":"School of Interactive Games and Media, Rochester Institute of Technology, Rochester, NY 14623, USA"}]},{"given":"Jeffrey T.","family":"Hansberger","sequence":"additional","affiliation":[{"name":"Army Research Laboratory, Huntsville, AL 35899, USA"}]}],"member":"1968","published-online":{"date-parts":[[2019,10,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1089\/109493101300117884","article-title":"Research on presence in virtual reality: A survey","volume":"4","author":"Schuemie","year":"2001","journal-title":"CyberPsychol. Behav."},{"unstructured":"Susi, T., Johannesson, M., and Backlund, P. (2007). Serious Games: An Overview, School of Humanities and Informatics, University of Sk\u00f6vde. Available online: http:\/\/urn.kb.se\/resolve?urn=urn:nbn:se:his:diva-1279.","key":"ref_2"},{"unstructured":"Djaouti, D., Alvarez, J., and Jessel, J.P. (2011). Classifying serious games: The G\/P\/S model. Handbook of Research on Improving Learning and Motivation through Educational Games: Multidisciplinary Approaches, IGI Global.","key":"ref_3"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1016\/j.entcom.2016.12.001","article-title":"Gamification of Apollo lunar exploration missions for learning engagement","volume":"19","author":"Peng","year":"2017","journal-title":"Entertain. Comput."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"624","DOI":"10.1016\/j.jesp.2009.02.016","article-title":"The development of the Game Engagement Questionnaire: A measure of engagement in video game-playing","volume":"45","author":"Brockmyer","year":"2009","journal-title":"J. Exp. Soc. Psychol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"769","DOI":"10.1016\/j.compedu.2010.10.020","article-title":"Educational virtual environments: A ten-year review of empirical research (1999\u20132009)","volume":"56","author":"Mikropoulos","year":"2011","journal-title":"Comput. Educ."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1016\/j.compedu.2016.02.002","article-title":"Virtual laboratories for education in science, technology, and engineering: A review","volume":"95","author":"Potkonjak","year":"2016","journal-title":"Comput. Educ."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"919","DOI":"10.1080\/0144929X.2016.1212931","article-title":"Virtual reality systems enhance students\u2019 achievements in engineering education","volume":"35","author":"Alhalabi","year":"2016","journal-title":"Behav. Inf. Technol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1515","DOI":"10.1007\/s10639-017-9676-0","article-title":"A review of the use of virtual reality head-mounted displays in education and training","volume":"23","author":"Jensen","year":"2018","journal-title":"Educ. Inf. Technol."},{"doi-asserted-by":"crossref","unstructured":"Freina, L., and Ott, M. (2015, January 23\u201324). A literature review on immersive virtual reality in education: State of the art and perspectives. Proceedings of the International Scientific Conference eLearning and Software for Education, Bucharest, Romania.","key":"ref_10","DOI":"10.12753\/2066-026X-15-020"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1016\/j.compedu.2006.04.004","article-title":"On the usability and likeability of virtual reality games for education: The case of VR-ENGAGE","volume":"50","author":"Virvou","year":"2008","journal-title":"Comput. Educ."},{"key":"ref_12","first-page":"298","article-title":"Applying situated learning in a virtual reality system to enhance learning motivation","volume":"1","author":"Mei","year":"2011","journal-title":"Int. J. Inf. Educ. Technol."},{"doi-asserted-by":"crossref","unstructured":"Cheng, A., Yang, L., and Andersen, E. (2017, January 6\u201311). Teaching language and culture with a virtual reality game. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA.","key":"ref_13","DOI":"10.1145\/3025453.3025857"},{"unstructured":"Adamo-Villani, N., and Wilbur, R.B. (2008, January 29\u201330). Effects of platform (immersive versus non-immersive) on usability and enjoyment of a virtual learning environment for deaf and hearing children. Proceedings of the Eurographics Symposium on Virtual Environments (EGVE 2008), Eindhoven, The Netherlands.","key":"ref_14"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1007\/s10055-016-0287-7","article-title":"A comparative evaluation of viewing metaphors on psychophysical skills education in an interactive virtual environment","volume":"20","author":"Parmar","year":"2016","journal-title":"Virtual Real."},{"key":"ref_16","first-page":"220","article-title":"Comparing Learning in Virtual Reality with Learning on a 2D Screen Using Electrostatics Activities","volume":"24","author":"Greenwald","year":"2018","journal-title":"J. UCS"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2045","DOI":"10.29333\/ejmste\/85874","article-title":"Mobile virtual reality as an educational platform: A pilot study on the impact of immersion and positive emotion induction in the learning process","volume":"14","author":"Contero","year":"2018","journal-title":"Eurasia J. Math. Sci. Technol. Educ."},{"doi-asserted-by":"crossref","unstructured":"Zizza, C., Starr, A., Hudson, D., Nuguri, S.S., Calyam, P., and He, Z. (2018, January 12\u201315). Towards a social virtual reality learning environment in high fidelity. Proceedings of the 2018 15th IEEE Annual Consumer Communications & Networking Conference (CCNC), Las Vegas, NV, USA.","key":"ref_18","DOI":"10.1109\/CCNC.2018.8319187"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1063","DOI":"10.1109\/TVCG.2017.2653117","article-title":"Effects of different types of virtual reality display on presence and learning in a safety training scenario","volume":"24","author":"Buttussi","year":"2017","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"404","DOI":"10.1162\/105474602760204309","article-title":"A survey of usability evaluation in virtual environments: Classification and comparison of methods","volume":"11","author":"Bowman","year":"2002","journal-title":"Presence Teleoper. Virtual Environ."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1080\/014492900750052679","article-title":"Evaluating the usability of virtual reality user interfaces","volume":"19","author":"Sutcliffe","year":"2000","journal-title":"Behav. Inf. Technol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"626","DOI":"10.1109\/TVCG.2012.43","article-title":"Evaluating display fidelity and interaction fidelity in a virtual reality game","volume":"18","author":"McMahan","year":"2012","journal-title":"IEEE Trans. Vis. Comput. Graph."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1016\/j.displa.2007.09.005","article-title":"Virtual reality induced symptoms and effects (VRISE): Comparison of head mounted display (HMD), desktop and projection display systems","volume":"29","author":"Sharples","year":"2008","journal-title":"Displays"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"740","DOI":"10.3102\/0034654315577210","article-title":"Gameplay engagement and learning in game-based learning: A systematic review","volume":"85","author":"Felicia","year":"2015","journal-title":"Rev. Educ. Res."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"767","DOI":"10.1177\/1046878113513079","article-title":"Subjective experience and sociability in a collaborative serious game","volume":"44","author":"Oksanen","year":"2013","journal-title":"Simul. Gaming"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1016\/j.chb.2014.12.052","article-title":"Audial engagement: Effects of game sound on learner engagement in digital game-based learning environments","volume":"46","author":"Byun","year":"2015","journal-title":"Comput. Hum. Behav."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1007\/s10758-015-9263-7","article-title":"What we can learn from the data: A multiple-case study examining behavior patterns by students with different characteristics in using a serious game","volume":"21","author":"Liu","year":"2016","journal-title":"Technol. Knowl. Learn."},{"doi-asserted-by":"crossref","unstructured":"Hookham, G., Nesbitt, K., and Kay-Lambkin, F. (2016, January 1\u20135). Comparing usability and engagement between a serious game and a traditional online program. Proceedings of the Australasian Computer Science Week Multiconference, Canberra, Australia.","key":"ref_28","DOI":"10.1145\/2843043.2843365"},{"doi-asserted-by":"crossref","unstructured":"Fernandez-Cervantes, V., and Stroutia, E. (2019, January 23\u201327). Virtual-Gym vR: A Virtual Reality Platform for Personalized Exergames. Proceedings of the 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), Osaka, Japan.","key":"ref_29","DOI":"10.1109\/VR.2019.8798298"},{"unstructured":"McMahan, R.P. (2011). Exploring the Effects of Higher-Fidelity Display and Interaction for Virtual Reality Games. [Ph.D. Thesis, Virginia Tech].","key":"ref_30"},{"unstructured":"National Science Foundation (2019, August 14). Percentage Distribution of Bachelor\u2019s Degrees Awarded, by Major Field Group: 1966\u20132010, Available online: https:\/\/www.nsf.gov\/statistics\/nsf13327\/pdf\/tab6.pdf.","key":"ref_31"},{"unstructured":"Markovich, S.J., and Chatzky, A. (2019, August 14). Space Exploration and U.S. Competitiveness. Available online: https:\/\/www.cfr.org\/backgrounder\/space-exploration-and-us-competitiveness.","key":"ref_32"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"29","DOI":"10.5926\/arepj1962.47.0_29","article-title":"Active learning: Cooperation in the classroom","volume":"47","author":"Johnson","year":"2008","journal-title":"Annu. Rep. Educ. Psychol. Jpn."},{"unstructured":"NASA\u2019s Jet Propulsion Laboratory (2019, August 14). NASA\u2019s Eyes, Available online: https:\/\/eyes.nasa.gov\/.","key":"ref_34"},{"unstructured":"NASA\u2019s Jet Propulsion Laboratory (2019, August 16). Space Place: Explore Earth and Space, Available online: https:\/\/spaceplace.nasa.gov\/.","key":"ref_35"},{"unstructured":"NASA (2019, August 16). NASA 3D Resources, Available online: https:\/\/nasa3d.arc.nasa.gov\/.","key":"ref_36"},{"unstructured":"Antoun, C. (2019, August 16). Station Spacewalk Game, Available online: https:\/\/www.nasa.gov\/multimedia\/3d_resources\/station_spacewalk_game.html.","key":"ref_37"},{"unstructured":"NASA\u2019s Jet Propulsion Laboratory and Goddard Space Flight Center (2019, August 16). Earth\u2019s Moon, Available online: https:\/\/moon.nasa.gov\/.","key":"ref_38"},{"unstructured":"Virtual Reality Education & Corporate Training (2019, August 16). Apollo 11 VR. Available online: https:\/\/immersivevreducation.com\/apollo-11-vr\/.","key":"ref_39"},{"unstructured":"Naval Aviation Museum (2019, August 16). Apollo 11 Virtual Reality. Available online: https:\/\/www.navalaviationmuseum.org\/attractions\/apollo-11-virtual-reality\/.","key":"ref_40"},{"doi-asserted-by":"crossref","unstructured":"Cao, L., Peng, C., and Hansberger, J. (2019, January 18\u201321). A Large Curved Display System in Virtual Reality for Immersive Data Interaction. Proceedings of the 2019 IEEE Games, Entertainment, Media Conference (GEM), New Haven, CT, USA.","key":"ref_41","DOI":"10.1109\/GEM.2019.8811550"}],"container-title":["Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2227-9709\/6\/4\/44\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:27:23Z","timestamp":1760189243000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2227-9709\/6\/4\/44"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,10,3]]},"references-count":41,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2019,12]]}},"alternative-id":["informatics6040044"],"URL":"https:\/\/doi.org\/10.3390\/informatics6040044","relation":{},"ISSN":["2227-9709"],"issn-type":[{"type":"electronic","value":"2227-9709"}],"subject":[],"published":{"date-parts":[[2019,10,3]]}}}