{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,2]],"date-time":"2026-07-02T17:34:11Z","timestamp":1783013651818,"version":"3.54.6"},"reference-count":56,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2020,4,5]],"date-time":"2020-04-05T00:00:00Z","timestamp":1586044800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Simulations, serious games, and virtual reality (SSG) applications represent promising support for achieving practical proficiency, but it is difficult to know how to introduce them into a new environment. This paper aims to contribute to a better understanding of introducing new SSGs to a non-computer related educational environment\u2014biomedical laboratory science (BLS) education. By following the choice, construction, and evaluation of a gamified app for practicing phlebotomy (StikkApp), not only the usefulness of the application, but also the general needs and possibilities for supporting SSG applications, are discussed. This paper presents the evaluation of StikkApp through an experimental study examining its use on mobile devices, as a web app and by discussing challenges for a corresponding virtual reality app by BLS students and their teachers. This evaluation focused on questions concerning usage scenarios, technologies, and how the design of the app can be aligned to learning goals necessary for education. By discussing these requirements and possibilities for apps and technology support for using SSG apps for BLS students, this paper contributes to a better understanding of using digital support for sustainable education.<\/jats:p>","DOI":"10.3390\/info11040195","type":"journal-article","created":{"date-parts":[[2020,4,6]],"date-time":"2020-04-06T05:05:13Z","timestamp":1586149513000},"page":"195","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":23,"title":["Games on Mobiles via Web or Virtual Reality Technologies: How to Support Learning for Biomedical Laboratory Science Education"],"prefix":"10.3390","volume":"11","author":[{"given":"Tord Hettervik","family":"Fr\u00f8land","sequence":"first","affiliation":[{"name":"Department of Safety, Chemistry and Biomedical Laboratory Sciences, Western Norway University of Applied Sciences, 5063 Bergen, Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1149-8820","authenticated-orcid":false,"given":"Ilona","family":"Heldal","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Electrical Engineering and Mathematical Sciences, Western Norway University of Applied Sciences, 5063 Bergen, Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gry","family":"Sj\u00f8holt","sequence":"additional","affiliation":[{"name":"Department of Safety, Chemistry and Biomedical Laboratory Sciences, Western Norway University of Applied Sciences, 5063 Bergen, Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9617-9329","authenticated-orcid":false,"given":"Elisabeth","family":"Ersv\u00e6r","sequence":"additional","affiliation":[{"name":"Department of Safety, Chemistry and Biomedical Laboratory Sciences, Western Norway University of Applied Sciences, 5063 Bergen, Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2020,4,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1519143","DOI":"10.1080\/2331186X.2018.1519143","article-title":"Digital competence and digital literacy in higher education research: Systematic review of concept use","volume":"5","author":"Spante","year":"2018","journal-title":"Cogent Educ."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Freina, L., and Ott, M. (2015, January 23\u201324). A literature review on immersive virtual reality in education: State of the art and perspectives. Proceedings of the International Scientific Conference eLearning and Software for Education, Bucharest, Romania.","DOI":"10.12753\/2066-026X-15-020"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1515","DOI":"10.1007\/s10639-017-9676-0","article-title":"A review of the use of virtual reality head-mounted displays in education and training","volume":"23","author":"Jensen","year":"2018","journal-title":"Educ. Inf. Technol."},{"key":"ref_4","unstructured":"Fomin, V.V., Heldal, I., and Wijkmark, C.H. (2018, January 18\u201320). Technology Adoption Failure Through the Prism of Organizational Regulation Model. Proceedings of the NOKOBIT\u2014Norsk Konferanse for Organisasjoners Bruk av IT, Svalbard, Norway."},{"key":"ref_5","first-page":"4","article-title":"The Digital HealthLab: Supporting Interdisciplinary Projects in Engineering and in Health Education","volume":"8","author":"Heldal","year":"2018","journal-title":"J. Appl. Tech. Educ. Sci."},{"key":"ref_6","unstructured":"Alklind Taylor, A.-S. (2014). Facilitation Matters: A Framework for Instructor-Led Serious Gaming. [Ph.D. Thesis, School of Informatics, University of Sk\u00f6vde]."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Scholz, R.W., Diefenbach, S., Franke, L., Grunwald, A., Helbing, D., Hill, R., Hilty, L., H\u00f6jer, M., Klauser, S., and Montag, C. (2018). Unintended side effects of the digital transition: European scientists\u2019 messages from a proposition-based expert round table. Sustainability, 10.","DOI":"10.3390\/su10062001"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"4797032","DOI":"10.1155\/2019\/4797032","article-title":"A meta-analysis of use of serious games in education over a decade","volume":"2019","author":"Zhonggen","year":"2019","journal-title":"Int. J. Comput. Games Technol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"741","DOI":"10.1007\/s11423-010-9183-0","article-title":"Measuring learning in serious games: A case study with structural assessment","volume":"59","author":"Wouters","year":"2011","journal-title":"Educ. Technol. Res. Dev."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"e12945","DOI":"10.2196\/12945","article-title":"Digital problem-based learning in health professions: Systematic review and meta-analysis by the Digital Health Education Collaboration","volume":"21","author":"Car","year":"2019","journal-title":"J. Med. Internet Res."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1111\/bjet.12223","article-title":"Benefits, barriers and guideline recommendations for the implementation of serious games in education for stakeholders and policymakers","volume":"47","author":"Tsekleves","year":"2016","journal-title":"Br. J. Educ. Technol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"178","DOI":"10.1016\/j.compedu.2015.11.003","article-title":"An update to the systematic literature review of empirical evidence of the impacts and outcomes of computer games and serious games","volume":"94","author":"Boyle","year":"2016","journal-title":"Comput. Educ."},{"key":"ref_13","unstructured":"(2020, April 03). Policy Document: The Norwegian Institute of Biomedical Science: Future Trends in Biomedical Laboratory Science: A Norwegian Perspective. Available online: https:\/\/www.nito.no\/fagmiljo\/bioingeniorfaglig-institutt\/brosjyrer-og-dokumenter\/in-english\/."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"e1","DOI":"10.1309\/LMZK40NMGLFLW9SX","article-title":"The American Society for Clinical Pathology\u2019s 2012 Vacancy survey of clinical laboratories in the United States","volume":"44","author":"Garcia","year":"2013","journal-title":"Lab. Med."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1162\/PRES_a_00135","article-title":"Predicting the efficacy of simulator-based training using a perceptual judgment task versus questionnaire-based measures of presence","volume":"22","author":"Wallis","year":"2013","journal-title":"Presence (Camb.)"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Froland, T., Elsv\u00e6r, E., Sj\u00f8holt, G., Heldal, I., Hjellbrekke, A., Logeswaran, S., Kovari, A., Katona, J., Costescu, C., and Rosan, A. (2019, January 23\u201325). mStikk\u2014A Mobile Application for Learning Phlebotomy. Proceedings of the 10th IEEE International Conference on Cognitive Infocommunications, Naples, Italy.","DOI":"10.1109\/CogInfoCom47531.2019.9089979"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Fr\u00f8land, T.H., Heldal, I., Ersv\u00e6r, E., Sj\u00f8holt, G., and K\u0151v\u00e1ri, A. (2019, January 21\u201323). wStikk\u2014Web-based Phlebotomy Learning. Proceedings of the 2019 E-Health and Bioengineering Conference (EHB), Iasi, Romania.","DOI":"10.1109\/EHB47216.2019.8970049"},{"key":"ref_18","unstructured":"Susi, T., Johannesson, M., and Backlund, P. (2007). Serious Games\u2014An Overview. IKI Technical Reports (HS-IKI-TR-07-001), School of Informatics, University of Sk\u00f6vde."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"603","DOI":"10.1162\/pres.1997.6.6.603","article-title":"A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments","volume":"6","author":"Slater","year":"1997","journal-title":"Presence Teleoperators Virtual Environ."},{"key":"ref_20","unstructured":"Heldal, I. (2004). The Usability of Collaborative Virtual Environments: Towards an Evaluation Framework. [Ph.D Thesis, Chalmers]."},{"key":"ref_21","unstructured":"(2020, April 03). Government Report: Digitalization for Education 2017\u20132021; [Eng: Digitaliseringstrategi for grunnoppl\u00e6ringen 2017\u20132021]; Gov. Report. Available online: https:\/\/www.regjeringen.no\/no\/dokumenter\/framtid-fornyelse-og-digitalisering\/id2568347."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"e182","DOI":"10.2196\/jmir.4346","article-title":"A Cost-Effectiveness Analysis of Blended Versus Face-to-Face Delivery of Evidence-Based Medicine to Medical Students","volume":"17","author":"Maloney","year":"2015","journal-title":"J. Med. Internet Res."},{"key":"ref_23","first-page":"1933","article-title":"Cost-Effectiveness of Digital Wound Care Education in a Healthcare Organization","volume":"264","author":"Hiltunen","year":"2019","journal-title":"Stud. Health Technol. Inform."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"e12937","DOI":"10.2196\/12937","article-title":"Mobile Digital Education for Health Professions: Systematic Review and Meta-Analysis by the Digital Health Education Collaboration","volume":"21","author":"Dunleavy","year":"2019","journal-title":"J. Med. Internet Res."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"e12967","DOI":"10.2196\/12967","article-title":"Effectiveness of Digital Education on Communication Skills Among Medical Students: Systematic Review and Meta-Analysis by the Digital Health Education Collaboration","volume":"21","author":"Kyaw","year":"2019","journal-title":"J. Med. Internet Res."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"e12959","DOI":"10.2196\/12959","article-title":"Virtual Reality for Health Professions Education: Systematic Review and Meta-Analysis by the Digital Health Education Collaboration","volume":"21","author":"Kyaw","year":"2019","journal-title":"J. Med. Internet. Res."},{"key":"ref_27","first-page":"6","article-title":"A problem based learning meta analysis: Differences across problem types, implementation types, disciplines, and assessment levels","volume":"3","author":"Walker","year":"2009","journal-title":"Interdiscip. J. Probl. -Based Learn."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1016\/j.im.2018.06.003","article-title":"Gamification: A key determinant of massive open online course (MOOC) success","volume":"56","author":"Aparicio","year":"2019","journal-title":"Inf. Manag."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"103701","DOI":"10.1016\/j.compedu.2019.103701","article-title":"Challenges in the online component of blended learning: A systematic review","volume":"144","author":"Rasheed","year":"2020","journal-title":"Comput. Educ."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1080\/00461520.2014.965823","article-title":"The ICAP framework: Linking cognitive engagement to active learning outcomes","volume":"49","author":"Chi","year":"2014","journal-title":"Educ. Psychol."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"435","DOI":"10.1016\/j.compedu.2011.07.015","article-title":"Serious games and learning effectiveness: The case of It\u2019sa Deal!","volume":"58","year":"2012","journal-title":"Comput. Educ."},{"key":"ref_32","first-page":"1","article-title":"Planning for neomillennial learning styles: Implications for investments in technology and faculty","volume":"5","author":"Dede","year":"2005","journal-title":"Educ. Net Gener."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"268","DOI":"10.1038\/499268a","article-title":"Education online: The virtual lab","volume":"499","author":"Waldrop","year":"2013","journal-title":"Nat. News"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1016\/j.iheduc.2004.02.001","article-title":"Blended learning: Uncovering its transformative potential in higher education","volume":"7","author":"Garrison","year":"2004","journal-title":"Internet High. Educ."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1002\/nop2.188","article-title":"Xperiences and perceptions of the theory-practice gap in nursing in a resource-constrained setting: A qualitative description study","volume":"6","author":"Salifu","year":"2019","journal-title":"Nurs. Open"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Backlund, P., Heldal, I., Engstr\u00f6m, H., Johannesson, M., and Lebram, M. (2013, January 20\u201322). Collaboration Patterns in Mixed Reality Environments for a New Emergency Training Center. Proceedings of the IEEE European Modelling Symposium, Manchester, UK.","DOI":"10.1109\/EMS.2013.81"},{"key":"ref_37","unstructured":"Amos, J.R., Bashir, R., Cross, K.J., Herman, G.L., Loewenstein, J., Pool, M.A., and Silverman, D. (2017, January 24\u201327). Defining the Frontiers of Bioengineering Education at Illinois and Beyond. Proceedings of the ASEE Annual Conference and Exposition, Conference Proceedings, Columbus, OH, USA."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"White, J.A., Gaver, D.P., Butera, R.J., Choi, B., Dunlop, M.J., Grande-Allen, K.J., Grosberg, A., Hitchcock, R.W., Huang-Saad, A.Y., and Kotche, M. (2020). Core Competencies for Undergraduates in Bioengineering and Biomedical Engineering: Findings, Consequences, and Recommendations, Springer.","DOI":"10.1007\/s10439-020-02468-2"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1007\/s10055-016-0298-4","article-title":"Success factors for serious games to enhance learning: A systematic review","volume":"21","author":"Ravyse","year":"2017","journal-title":"Virtual Real."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Backlund, P., Engstr\u00f6m, H., Marklund, B.B., and Toftedahl, M. (2017, January 6\u20138). Developing games for non-leisure contexts: Identification of challenges and research gaps. in Virtual Worlds and Games for Serious Applications (VS-Games). Proceedings of the 2017 9th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games), Athens, Greece.","DOI":"10.1109\/VS-GAMES.2017.8055806"},{"key":"ref_41","unstructured":"Heldal, I., and Wijkmark, C.H. (2015, January 9\u201311). Facilitators and Obstacles in the Introduction and Use of Technology: Simulation and Serious Games as Support in Firefighter Training. Proceedings of the 9th International Conference on Researching Work and Learning, Singapore."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"750","DOI":"10.1515\/CCLM.2006.123","article-title":"Errors in clinical laboratories or errors in laboratory medicine?","volume":"44","author":"Plebani","year":"2006","journal-title":"Clin. Chem. Lab. Med. (CCLM)"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1095","DOI":"10.1111\/j.1365-2929.2004.01959.x","article-title":"Simulation and clinical practice: Strengthening the relationship","volume":"38","author":"Kneebone","year":"2004","journal-title":"Med. Educ."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1109\/MEMB.2003.1237490","article-title":"Constructing shareable learning materials in bioengineering education","volume":"22","author":"Brophy","year":"2003","journal-title":"IEEE Eng. Med. Biol. Mag."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Sj\u00f8holt, G., Ryningen, A., Gundersen, L.B., Rostad, K., and Ersv\u00e6r, E. (2019, January 11\u201313). Open Online E-Learning Resources at epraksis.no as preparation for hands-on laboratory practice. Proceedings of the International Technology, Education and Development Conference (INTED2019), Seville, Spain.","DOI":"10.21125\/inted.2019.1628"},{"key":"ref_46","unstructured":"(2020, April 03). BioDigi-Project. Available online: https:\/\/www.tuas.fi\/en\/research-and-development\/projects\/biodigi-digital-online-portal-biomedical-laborator\/."},{"key":"ref_47","unstructured":"(2020, April 03). FutureEdu. Available online: https:\/\/futureedu.savonia.fi\/."},{"key":"ref_48","unstructured":"(2020, April 03). Labster. Available online: https:\/\/www.labster.com\/."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1309\/LMVB30QRE3AIEUXE","article-title":"Best Practices in Undergraduate Clinical Laboratory Science Online Education and Effective Use of Educational Technology Tools","volume":"43","author":"Hammerling","year":"2012","journal-title":"Lab. Med."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Gardner, A., Duprez, W., Stauffer, S., Ungu, D.A.K., and Clauson-Kaas, F. (2019). Labster Virtual Lab Experiments: Basic Biochemistry, Springer.","DOI":"10.1007\/978-3-662-58499-6"},{"key":"ref_51","unstructured":"Kalles, D. (2017). Simulation for Blended-Learning Laboratory Education, The Envisioning Report for Empowering Universities."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"M\u00e4eots, M., Siiman, L., and Pedaste, M. (2014). Designing Interactive Scratch Content for Future E-books. Lecture Notes in Computer Science, Proceedings of the International Conference on Web-Based Learning, Tallinn, Estonia, 14\u201317 August 2014, Springer.","DOI":"10.1007\/978-3-319-13296-9_26"},{"key":"ref_53","unstructured":"Oates, B.J. (2005). Researching Information Systems and Computing, Sage."},{"key":"ref_54","first-page":"1995","article-title":"Technology readiness levels","volume":"6","author":"Mankins","year":"1995","journal-title":"White Paper"},{"key":"ref_55","unstructured":"Hus\u00f8y, A. (2012). Blodpr\u00f8vetaking i Praksis. 2. Utgave, Cappelen Damm Akademisk."},{"key":"ref_56","unstructured":"Ersv\u00e6r, E., and Sj\u00f8holt, G. (2019, July 19). ePraksis. 2019 Learning Resource for Biomedical Laboratory Science. Available online: https:\/\/epraksis.no."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/11\/4\/195\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:15:46Z","timestamp":1760174146000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/11\/4\/195"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,4,5]]},"references-count":56,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2020,4]]}},"alternative-id":["info11040195"],"URL":"https:\/\/doi.org\/10.3390\/info11040195","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,4,5]]}}}