{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,22]],"date-time":"2026-07-22T18:30:43Z","timestamp":1784745043641,"version":"3.55.0"},"reference-count":44,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2022,6,29]],"date-time":"2022-06-29T00:00:00Z","timestamp":1656460800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Informatics"],"abstract":"<jats:p>With concepts such as immersion and presence, known as hedonistic qualities of importance in the virtual reality (VR) experience, the question arises whether the more pragmatic heuristics are effective in the evaluation of an artifact. However, despite the importance of the heuristics for artifact evaluations, the available studies do not provide a rigorous review of these heuristics and their efficiency. Thus, this review aims to look at how heuristics have been applied in various virtual learning environments (VLEs) that involve virtual reality learning activities, either for use in heuristic evaluations or as design principles. In addition, it examines how these heuristics support the evaluation of a more hedonistic quality, such as presence, and lastly, the aim is to gauge the estimated efficiency of heuristics as an evaluation method. This article is a systematic review of research investigating using heuristics in a virtual reality learning context. The review includes articles published from January 2017 to February 2022, and from the screened records, twelve articles were analyzed in full-text form. This review shows the versatility of heuristics and their applications as well as the key concepts that are vital to the user\u2019s experience in a virtual reality learning context. This review indicates that heuristic evaluation is a valuable tool, as it provides a clear summary of what needs to be handled in the next iteration of the application.<\/jats:p>","DOI":"10.3390\/informatics9030051","type":"journal-article","created":{"date-parts":[[2022,6,29]],"date-time":"2022-06-29T22:43:28Z","timestamp":1656542608000},"page":"51","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Analyzing the Use of Heuristics in a Virtual Reality Learning Context: A Literature Review"],"prefix":"10.3390","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0682-3799","authenticated-orcid":false,"given":"Abdulghafour","family":"Mohammad","sequence":"first","affiliation":[{"name":"School of Business, Economics and IT, University West, 46186 Trollhattan, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Line","family":"Pedersen","sequence":"additional","affiliation":[{"name":"School of Business, Economics and IT, University West, 46186 Trollhattan, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,6,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Kurniawan, C., Rosmansyah, Y., and Dabarsyah, B. (2019, January 25\u201326). A systematic literature review on virtual reality for learning. Proceedings of the 2019 IEEE 5th International Conference on Wireless and Telematics (ICWT), Yogyakarta, Indonesia.","DOI":"10.1109\/ICWT47785.2019.8978263"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Gigante, M.A. (1993). Virtual reality: Definitions, history and applications. Virtual Reality Systems, Academic Press.","DOI":"10.1016\/B978-0-12-227748-1.50009-3"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1109\/TPC.2018.2804238","article-title":"Understanding virtual reality: Presence, embodiment, and professional practice","volume":"61","author":"Tham","year":"2018","journal-title":"IEEE Trans. Prof. Commun."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1080\/17439880500250527","article-title":"Virtual learning environments: Three implementation perspectives","volume":"30","author":"Keller","year":"2005","journal-title":"Learn. Media Technol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2086","DOI":"10.3389\/fpsyg.2018.02086","article-title":"The past, present, and future of virtual and augmented reality research: A network and cluster analysis of the literature","volume":"9","author":"Cipresso","year":"2018","journal-title":"Front. Psychol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"e05519","DOI":"10.1016\/j.heliyon.2020.e05519","article-title":"Understanding satisfaction essentials of E-learning in higher education: A multi-generational cohort perspective","volume":"6","author":"Yawson","year":"2020","journal-title":"Heliyon"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Pinto, R.D., Peixoto, B., Melo, M., Cabral, L., and Bessa, M. (2021). Foreign language learning gamification using Virtual Reality\u2014A systematic review of empirical research. Educ. Sci., 11.","DOI":"10.3390\/educsci11050222"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Vergara, D., Extremera, J., Rubio, M.P., and Davila, L.P. (2020). The technological obsolescence of virtual reality learning environments. Appl. Sci., 10.","DOI":"10.3390\/app10030915"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Vergara-Rodr\u00edguez, D., G\u00f3mez-Asenjo, A., Fern\u00e1ndez-Arias, P., G\u00f3mez-Vallecillo, A.I., Lamas-\u00c1lvarez, V.E., and de La Iglesia, C.D.S. (October, January 30). Immersive vs. non-immersive virtual reality learning environments. Proceedings of the 2021 XI International Conference on Virtual Campus (JICV), Salamanca, Spain.","DOI":"10.1109\/JICV53222.2021.9600337"},{"key":"ref_10","unstructured":"Preece, J., Rogers, Y., and Sharp, H. (2016). Interaktionsdesign: Bortom M\u00e4nniska-Dator-Interaktion, Studentlitteratur."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Grivokostopoulou, F., Perikos, I., and Hatzilygeroudis, I. (2016, January 2\u20134). An innovative educational environment based on virtual reality and gamification for learning search algorithms. Proceedings of the 2016 IEEE Eighth International Conference on Technology for Education (T4E), Mumbai, India.","DOI":"10.1109\/T4E.2016.029"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1016\/j.cag.2005.10.004","article-title":"Virtual reality and mixed reality for virtual learning environments","volume":"30","author":"Pan","year":"2006","journal-title":"Comput. Graph."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"674179","DOI":"10.3389\/fpsyg.2021.674179","article-title":"Embodiment in Virtual Reality Intensifies Emotional Responses to Virtual Stimuli","volume":"12","author":"Gall","year":"2021","journal-title":"Front. Psychol."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Fernandes, F., and Werner, C. (2019, January 28\u201331). Towards Immersive Learning in Object-Oriented Paradigm: A Preliminary Study. Proceedings of the 2019 21st Symposium on Virtual and Augmented Reality (SVR), Rio de Janeiro, Brazil.","DOI":"10.1109\/SVR.2019.00026"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1016\/j.chb.2017.09.012","article-title":"Empathy and embodied experience in virtual environment: To what extent can virtual reality stimulate empathy and embodied experience?","volume":"78","author":"Shin","year":"2018","journal-title":"Comput. Hum. Behav."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1826","DOI":"10.1016\/j.tele.2017.05.013","article-title":"The role of affordance in the experience of virtual reality learning: Technological and affective affordances in virtual reality","volume":"34","author":"Shin","year":"2017","journal-title":"Telemat. Inform."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"132","DOI":"10.1016\/j.csi.2019.03.003","article-title":"Developing usability heuristics with PROMETHEUS: A case study in virtual learning environments","volume":"65","author":"Figueroa","year":"2019","journal-title":"Comput. Stand. Interfaces"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"e55","DOI":"10.1093\/jamia\/ocw100","article-title":"Comparison of heuristic and cognitive walkthrough usability evaluation methods for evaluating health information systems","volume":"24","author":"Khajouei","year":"2017","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_19","unstructured":"Joyce, A. (2021, December 29). 10 Usability Heuristics Applied to Virtual Reality. NN\/g Nielsen Norman Group. 11 July 2021. Available online: https:\/\/www.nngroup.com\/articles\/usability-heuristics-virtual-reality\/."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1207\/S15327590IJHC1304_03","article-title":"Criteria for Evaluating Usability Evaluation Methods","volume":"13","author":"Hartson","year":"2001","journal-title":"Int. J. Hum. Comput. Interact."},{"key":"ref_21","unstructured":"Wilson, C. (2013). User Interface Inspection Methods: A User-Centered Design Method, Newnes."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Nielsen, J., and Molich, R. (1990, January 1\u20135). Heuristic evaluation of user interfaces. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Seattle, WA, USA.","DOI":"10.1145\/97243.97281"},{"key":"ref_23","unstructured":"Nielsen, J. (2021, December 29). 10 Usability Heuristics for User Interface Design. Available online: https:\/\/www.nngroup.com\/articles\/ten-usability-heuristics\/."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Gammanpila, A.C., Perera, V.A., Senaratna, H.A., Edirisinghe, E.W., Manawadu, U.A., and De Silva, P.R. (2019, January 2\u20135). Virtual Reality for Learning: Assessment of Awareness and Preference in Emerging Regions. Proceedings of the 2019 19th International Conference on Advances in ICT for Emerging Regions (ICTer), Colombo, Sri Lanka.","DOI":"10.1109\/ICTer48817.2019.9023729"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Jim\u00e9nez, C., Rusu, C., Roncagliolo, S., Inostroza, R., and Rusu, V. (2012, January 12\u201316). Evaluating a methodology to establish usability heuristics. Proceedings of the 2012 31st International Conference of the Chilean Computer Science Society, Valparaiso, Chile.","DOI":"10.1109\/SCCC.2012.14"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"b2535","DOI":"10.1136\/bmj.b2535","article-title":"Preferred reporting items for systematic reviews and metaanalyses: The PRISMA statement","volume":"339","author":"Moher","year":"2009","journal-title":"BMJ"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"104846","DOI":"10.1016\/j.nedt.2021.104846","article-title":"Design and evaluation of a 3D serious game for communication learning in nursing education","volume":"100","author":"Hara","year":"2021","journal-title":"Nurse Educ. Today"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"904","DOI":"10.3897\/jucs.2020.049","article-title":"Exploring virtual reality in the higher education classroom: Using VR to build knowledge and understanding","volume":"26","author":"Young","year":"2020","journal-title":"J. Univers. Comput. Sci."},{"key":"ref_29","unstructured":"Benson, L., Elliott, D., Grant, M., Holschuh, D., Kim, B., Kim, H., and Reeves, T.C. (2002). Usability and Instructional Design Heuristics for E-Learning Evaluation, Association for the Advancement of Computing in Education (AACE)."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Mtebe, J.S., and Kissaka, M.M. (2015, January 6\u20138). Heuristics for evaluating usability of learning management systems in Africa. Proceedings of the 2015 IST-Africa Conference, Lilongwe, Malawi.","DOI":"10.1109\/ISTAFRICA.2015.7190521"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"e1983","DOI":"10.1002\/cav.1983","article-title":"Experts\u2019 review of a virtual environment for virtual clinical simulation in South Africa","volume":"32","author":"Botha","year":"2021","journal-title":"Comput. Animat. Virtual Worlds"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"394","DOI":"10.1007\/s11528-020-00572-7","article-title":"Approaches to Integrate Virtual Reality into K-16 Lesson Plans: An Introduction for Teachers","volume":"65","author":"Hayes","year":"2021","journal-title":"TechTrends"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"e20797","DOI":"10.2196\/20797","article-title":"Exploring user needs in the development of a virtual reality\u2013based advanced life support training platform: Exploratory usability study","volume":"8","author":"Moore","year":"2020","journal-title":"JMIR Serious Games"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"831","DOI":"10.1016\/j.intcom.2004.05.001","article-title":"Heuristic evaluation of virtual reality applications","volume":"16","author":"Sutcliffe","year":"2004","journal-title":"Interact. Comput."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Oliveira, E., Sim\u00f5es, F.P., and Correia, W.F. (2017, January 1\u20134). Heuristics Evaluation and Improvements for Low-Cost Virtual Reality. Proceedings of the 2017 19th Symposium on Virtual and Augmented Reality (SVR), Curitiba, Brazil.","DOI":"10.1109\/SVR.2017.31"},{"key":"ref_36","unstructured":"Rusu, C., Mu\u00f1oz, R., Roncagliolo, S., Rudloff, S., Rusu, V., and Figueroa, A. (2011, January 21). Usability heuristics for virtual worlds. Proceedings of the Third International Conference on Advances in Future Internet, Nice\/Saint Laurent du Var, France."},{"key":"ref_37","unstructured":"Shneiderman, B., and Plaisant, C. (2010). Designing the User Interface: Strategies for Effective Human-Computer Interaction, Pearson Education."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"332","DOI":"10.1038\/nrn1651","article-title":"From presence to consciousness through virtual reality","volume":"6","author":"Slater","year":"2005","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1111\/j.1468-2885.2004.tb00302.x","article-title":"Presence, explicated","volume":"14","author":"Lee","year":"2004","journal-title":"Commun. Theory"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1016\/j.chb.2018.04.019","article-title":"The state of immersive technology research: A literature analysis","volume":"86","author":"Suh","year":"2018","journal-title":"Comput. Hum. Behav."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1016\/j.intcom.2012.03.004","article-title":"Subjective and behavioral presence measurement and interactivity in the collaborative augmented reality game TimeWarp","volume":"24","author":"Klatt","year":"2012","journal-title":"Interact. Comput."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"212","DOI":"10.1016\/j.chb.2016.03.094","article-title":"Teaching training in a mixed-reality integrated learning environment","volume":"62","author":"Ke","year":"2016","journal-title":"Comput. Hum. Behav."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1162\/PRES_a_00124","article-title":"The sense of embodiment in virtual reality","volume":"21","author":"Kilteni","year":"2012","journal-title":"Presence Teleoperators Virtual Environ."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1145\/142413.142416","article-title":"Research directions in virtual environments","volume":"26","author":"Brooks","year":"1992","journal-title":"Comput. Graph."}],"container-title":["Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2227-9709\/9\/3\/51\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:40:42Z","timestamp":1760139642000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2227-9709\/9\/3\/51"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,6,29]]},"references-count":44,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["informatics9030051"],"URL":"https:\/\/doi.org\/10.3390\/informatics9030051","relation":{},"ISSN":["2227-9709"],"issn-type":[{"value":"2227-9709","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,6,29]]}}}