{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:53:15Z","timestamp":1760147595141,"version":"build-2065373602"},"reference-count":110,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2023,2,19]],"date-time":"2023-02-19T00:00:00Z","timestamp":1676764800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>As gamification spreads to new areas, new applications are being developed and the interest in evaluating gamified systems continues to grow. To date, however, no one has comprehensively approached this topic: multiple evaluation dimensions and measures have been proposed and applied without any effort to organize them into a full gamut of tools for the multi-dimensional evaluation of gamified systems. This paper addresses this gap by proposing GATUGU, a set of six perspectives of evaluation of gamified systems: General effects of gamification, Area-specific effects of gamification, Technical quality of gamified systems, Use of gamified systems, Gamefulness of gamified systems, and User experience of gamified systems. For each perspective, GATUGU indicates the relevant dimensions of evaluation, and, for each dimension, one measure is suggested. GATUGU does not introduce any new measurement tools but merely recommends one of the available tools for each dimension, considering their popularity and ease of use. GATUGU can guide researchers in selecting gamification system evaluation perspectives and dimensions and in finding adequate measurement tools. Thanks to conforming to GATUGU, the published gamification system evaluation results will become easier to compare and to perform various kinds of meta-analyses on them.<\/jats:p>","DOI":"10.3390\/info14020136","type":"journal-article","created":{"date-parts":[[2023,2,20]],"date-time":"2023-02-20T01:36:37Z","timestamp":1676856997000},"page":"136","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["GATUGU: Six Perspectives of Evaluation of Gamified Systems"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2214-6989","authenticated-orcid":false,"given":"Jakub","family":"Swacha","sequence":"first","affiliation":[{"name":"Department of Information Technology in Management, University of Szczecin, 71-004 Szczecin, Poland"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1985-6285","authenticated-orcid":false,"given":"Ricardo","family":"Queir\u00f3s","sequence":"additional","affiliation":[{"name":"CRACS\u2014INESC Porto LA, 4169-007 Porto, Portugal"},{"name":"uniMAD\u2014ESMAD, Polytechnic of Porto, 4480-876 Vila do Conde, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0394-0527","authenticated-orcid":false,"given":"Jos\u00e9 Carlos","family":"Paiva","sequence":"additional","affiliation":[{"name":"CRACS\u2014INESC Porto LA, 4169-007 Porto, Portugal"},{"name":"Department of Computer Science, Faculty of Sciences, University of Porto, 4169-007 Porto, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,19]]},"reference":[{"key":"ref_1","unstructured":"Ouellette, M. (2016). Examining the Evolution of Gaming and Its Impact on Social, Cultural, and Political Perspectives, IGI Global."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1007\/978-3-319-39583-8_17","article-title":"The Bright and Dark Sides of Gamification","volume":"Volume 9684","author":"Andrade","year":"2016","journal-title":"Proceedings of the 13th International Conference on Intelligent Tutoring Systems"},{"key":"ref_3","unstructured":"Guarda, T., Portela, F., and Augusto, M.F. The Dangers of Gamification. Proceedings of the Advanced Research in Technologies, Information, Innovation and Sustainability."},{"key":"ref_4","unstructured":"Crawford, C. (1984). The Art of Computer Game Design, McGraw-Hill, Inc."},{"key":"ref_5","unstructured":"Julius, K., and Salo, J. (2013). Designing Gamification. [Master\u2019s Thesis, University of Oulu]."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/j.ijhcs.2014.09.006","article-title":"Gamification in theory and action: A survey","volume":"74","author":"Seaborn","year":"2015","journal-title":"Int. J. Hum. Comput. Stud."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Hamari, J., Koivisto, J., and Sarsa, H. (2014, January 6\u20139). Does Gamification Work?\u2014A Literature Review of Empirical studies on Gamification. Proceedings of the 47th Hawaii International Conference on System Sciences, Washington, DC, USA.","DOI":"10.1109\/HICSS.2014.377"},{"key":"ref_8","unstructured":"Hunicke, R., LeBlanc, M., and Zubek, R. (2004, January 26). MDA: A formal approach to game design and game research. Proceedings of the AAAI Workshop on Challenges in Game AI, San Jose, CA, USA."},{"key":"ref_9","unstructured":"Chou, Y.-k. (2022, September 26). Octalysis: Complete Gamification Framework. Yu-kai Chou: Gamification & Behavioral Design. Available online: https:\/\/yukaichou.com\/gamification-examples\/octalysis-complete-gamification-framework\/."},{"key":"ref_10","unstructured":"Marczewski, A. (2022, September 26). GAME: A Design Process Framework. Gamified U. Available online: https:\/\/www.gamified.uk\/2014\/05\/07\/game-design-process-framework\/."},{"key":"ref_11","unstructured":"Werbach, K., and Hunter, D. (2012). For the Win: How Game Thinking Can Revolutionize Your Business, Wharton Digital Press."},{"key":"ref_12","first-page":"5:1","article-title":"An Open-Source Gamified Programming Learning Environment","volume":"Volume 91","author":"Henriques","year":"2021","journal-title":"Proceedings of the Second International Computer Programming Education Conference (ICPEC 2021)"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Yigitbas, E., Schmidt, M., Bucchiarone, A., Gottschalk, S., and Engels, G. (2022, January 23\u201328). Gamification-Based UML Learning Environment in Virtual Reality. Proceedings of the 25th International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings, Montreal, QB, Canada.","DOI":"10.1145\/3550356.3559088"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Lee, C.S., Foo, J.J., Sangar, V.J., Chan, P.Y., Hor, W.K., and Chan, E.K. (2017, January 26\u201329). A Knowledge Management-Extended Gamified Customer Relationship Management System. Proceedings of the 2017 International Conference on Soft Computing, Intelligent System and Information Technology (ICSIIT), Bali, Indonesia.","DOI":"10.1109\/ICSIIT.2017.46"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Uden, L., Lu, W., and Ting, I.H. (2017). Knowledge Management in Organizations, Springer International Publishing.","DOI":"10.1007\/978-3-319-62698-7"},{"key":"ref_16","unstructured":"Herzig, P., Strahringer, S., and Ameling, M. (2012). Multikonferenz Wirtschaftsinformatik, GITO."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1016\/j.chb.2015.12.018","article-title":"Design and evaluation of a gamified system for ERP training","volume":"58","author":"Alcivar","year":"2016","journal-title":"Comput. Hum. Behav."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Jurado, J.L., Fernandez, A., and Collazos, C.A. (2015, January 21\u201323). Applying Gamification in the Context of Knowledge Management. Proceedings of the 15th International Conference on Knowledge Technologies and Data-Driven Business, Graz, Austria.","DOI":"10.1145\/2809563.2809606"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Elm, D., Kappen, D.L., Tondello, G.F., and Nacke, L.E. (2016, January 2\u20135). CLEVER: Gamification and Enterprise Knowledge Learning. Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts, Bremen, Germany.","DOI":"10.1145\/2968120.2987745"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Herrlich, M., Malaka, R., and Masuch, M. (2012, January 26\u201329). Mobile Application for Noise Pollution Monitoring through Gamification Techniques. Proceedings of the Entertainment Computing\u2014ICEC 2012, Bremen, Germany.","DOI":"10.1007\/978-3-642-33542-6"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1016\/j.anai.2020.07.018","article-title":"Developing and evaluating ASTHMAXcel adventures: A novel gamified mobile application for pediatric patients with asthma","volume":"125","author":"Hsia","year":"2020","journal-title":"Ann. Allergy Asthma Immunol."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Shortt, M., Tilak, S., Kuznetcova, I., Martens, B., and Akinkuolie, B. (2021). Gamification in mobile-assisted language learning: A systematic review of Duolingo literature from public release of 2012 to early 2020. Comput. Assist. Lang. Learn., 1\u201338.","DOI":"10.1080\/09588221.2021.1933540"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"24","DOI":"10.21608\/ajccr.2021.211511","article-title":"Investigating the Effect of Gamification on Website Features in E-Banking Sector: An Empirical Research","volume":"1","author":"Emam","year":"2021","journal-title":"Acad. J. Contemp. Commer. Res."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"29339","DOI":"10.1007\/s11042-021-11149-8","article-title":"Understanding consumer behavior in the multimedia context: Incorporating gamification in VR-enhanced web system for tourism e-commerce","volume":"80","author":"Jang","year":"2021","journal-title":"Multimed. Tools Appl."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Morschheuser, B., Hamari, J., Werder, K., and Abe, J. (2017, January 4\u20137). How to Gamify? A Method For Designing Gamification. Proceedings of the 50th Hawaii International Conference on System Sciences, HICSS, Waikoloa Village, HI, USA.","DOI":"10.24251\/HICSS.2017.155"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Tlili, A., and Chang, M. (2019). Data Analytics Approaches in Educational Games and Gamification Systems, Springer.","DOI":"10.1007\/978-981-32-9335-9"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Dubois, D.J., and Tamburrelli, G. (2013, January 18\u201326). Understanding gamification mechanisms for software development. Proceedings of the 9th Joint Meeting on Foundations of Software Engineering, Saint Petersburg, Russia.","DOI":"10.1145\/2491411.2494589"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Dal Sasso, T., Mocci, A., Lanza, M., and Mastrodicasa, E. (2017, January 20\u201324). How to gamify software engineering. Proceedings of the 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER), Klagenfurt, Austria.","DOI":"10.1109\/SANER.2017.7884627"},{"key":"ref_29","unstructured":"Fitz-Walter, Z.J. (2015). Achievement Unlocked: Investigating the Design of Effective Gamification Experiences for Mobile Applications Furthermore, Devices. [Ph.D. Thesis, Queensland University of Technology]."},{"key":"ref_30","unstructured":"Monteiro, R.H.B., Souza, M.R.D.A., Oliveira, S.R.B., Portela, C.D.S., and Lobato, C.E.D.C. (2021). The Diversity of Gamification Evaluation in the Software Engineering Education and Industry: Trends, Comparisons and Gaps. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Klock, A.C.T., Ogawa, A.N., Gasparini, I., and Pimenta, M.S. (2018, January 9\u201313). Does gamification matter?: A systematic mapping about the evaluation of gamification in educational environments. Proceedings of the 33rd Annual ACM Symposium on Applied Computing, Pau, France.","DOI":"10.1145\/3167132.3167347"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"450","DOI":"10.1016\/j.chb.2016.11.062","article-title":"The maturing of gamification research","volume":"71","author":"Nacke","year":"2017","journal-title":"Comput. Hum. Behav."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"122","DOI":"10.1049\/iet-sen.2018.5084","article-title":"Proposal of an assessment framework for gamified environments: A case study","volume":"13","year":"2019","journal-title":"IET Softw."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Ren, W., Barrett, S., and Das, S. (2020, January 17\u201319). Toward Gamification to Software Engineering and Contribution of Software Engineer. Proceedings of the 2020 4th International Conference on Management Engineering, Software Engineering and Service Sciences, Wuhan, China.","DOI":"10.1145\/3380625.3380628"},{"key":"ref_35","unstructured":"Inocencio, F. (2018, January 13\u201316). Using gamification in education: A systematic literature review. Proceedings of the Thirty Ninth International Conference on Information Systems, San Francisco, CA, USA."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1011","DOI":"10.25300\/MISQ\/2017\/41.4.01","article-title":"Toward Meaningful Engagement: A Framework for Design and Research of Gamified Information Systems","volume":"41","author":"Liu","year":"2017","journal-title":"MIS Q."},{"key":"ref_37","unstructured":"Marczewski, A. (2022, September 26). Defining Gamification\u2014What Do People Really Think? Gamified UK. Available online: http:\/\/www.gamified.uk\/2014\/04\/16\/defining-gamification-people-really-think."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1626","DOI":"10.1002\/cae.21992","article-title":"A comparative study on gamification of the flipped classroom in engineering education to enhance the effects of learning","volume":"26","author":"Jo","year":"2018","journal-title":"Comput. Appl. Eng. Educ."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1007\/s10758-020-09466-2","article-title":"Gamification to Enhance Motivation and Engagement in Blended eLearning for Technical and Vocational Education and Training","volume":"27","author":"Jayalath","year":"2022","journal-title":"Technol. Knowl. Learn."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"e04972","DOI":"10.1016\/j.heliyon.2020.e04972","article-title":"Gamification in engineering education\u2014An empirical assessment on learning and game performance","volume":"6","year":"2020","journal-title":"Heliyon"},{"key":"ref_41","first-page":"24","article-title":"Individuals, teams and organizations score with gamification","volume":"23","author":"Sarangi","year":"2015","journal-title":"Hum. Resour. Manag. Int. Dig."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"993","DOI":"10.1142\/S021819401550014X","article-title":"Working and Playing with Scrum","volume":"25","author":"Medeiros","year":"2015","journal-title":"Int. J. Softw. Eng. Knowl. Eng."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"508","DOI":"10.1016\/j.chb.2015.08.008","article-title":"Gamification of task performance with leaderboards: A goal setting experiment","volume":"71","author":"Landers","year":"2017","journal-title":"Comput. Hum. Behav."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1016\/j.ijresmar.2019.02.003","article-title":"Hook vs. hope: How to enhance customer engagement through gamification","volume":"36","author":"Eisingerich","year":"2019","journal-title":"Int. J. Res. Mark."},{"key":"ref_45","first-page":"1","article-title":"Investigating the Effect of Gamification Mechanics on Customer Loyalty in Online Stores","volume":"11","author":"Fathian","year":"2019","journal-title":"J. Inf. Technol. Manag."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Tan, Y., Shi, Y., Buarque, F., Gelbukh, A., Das, S., and Engelbrecht, A. (2015, January 25\u201328). A Gamified Online Forum Inspiring Group Intelligence Distillation for Policy Making. Proceedings of the Advances in Swarm and Computational Intelligence, Beijing, China.","DOI":"10.1007\/978-3-319-20466-6"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.chb.2018.09.022","article-title":"Applying gamifications to asynchronous online discussions: A mixed methods study","volume":"91","author":"Ding","year":"2019","journal-title":"Comput. Hum. Behav."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Stephanidis, C. (2018, January 15\u201320). Stress-Mentor: Linking Gamification and Behavior Change Theory in a Stress Management Application. Proceedings of the HCI International 2018\u2014Posters\u2019 Extended Abstracts, Las Vegas, NV, USA.","DOI":"10.1007\/978-3-319-92285-0"},{"key":"ref_49","unstructured":"Gao, Q., and Zhou, J. (2020, January 19\u201324). A Study on the Effect of Gamification on Alleviation Anxiety Levels of the Elderly in China. Proceedings of the Human Aspects of IT for the Aged Population. Healthy and Active Aging, Copenhagen, Denmark."},{"key":"ref_50","unstructured":"(2001). Software Engineering\u2014Product Quality (Standard No. ISO\/IEC 9126)."},{"key":"ref_51","unstructured":"Prisacaru, D. (2022, September 26). Performance, Reliability, Availability, Scalability. IASA. Available online: https:\/\/itabok.iasaglobal.org\/itabok3_0-2\/performance-reliability-availability-scalability\/."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"619","DOI":"10.1007\/s11257-019-09223-w","article-title":"Gameful Experience Questionnaire (GAMEFULQUEST): An instrument for measuring the perceived gamefulness of system use","volume":"29","author":"Hamari","year":"2019","journal-title":"User Model. User Adapt. Interact."},{"key":"ref_53","first-page":"224","article-title":"Comparison of Four TAM Item Formats: Effect of Response Option Labels and Order","volume":"14","author":"Lewis","year":"2019","journal-title":"J. Usability Stud."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"319","DOI":"10.2307\/249008","article-title":"Perceived usefulness, perceived ease of use, and user acceptance of information technology","volume":"13","author":"Davis","year":"1989","journal-title":"MIS Q."},{"key":"ref_55","unstructured":"Reeve, J. (2017). Understanding Motivation and Emotion, John Wiley & Sons. [7th ed.]."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1023\/A:1005614228250","article-title":"On the Assessment of Situational Intrinsic and Extrinsic Motivation: The Situational Motivation Scale (SIMS)","volume":"24","author":"Guay","year":"2000","journal-title":"Motiv. Emot."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1037\/0003-066X.55.1.68","article-title":"Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being","volume":"55","author":"Ryan","year":"2000","journal-title":"Am. Psychol."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"106963","DOI":"10.1016\/j.chb.2021.106963","article-title":"Revealing the theoretical basis of gamification: A systematic review and analysis of theory in research on gamification, serious games and game-based learning","volume":"125","author":"Krath","year":"2021","journal-title":"Comput. Hum. Behav."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"692","DOI":"10.2307\/256287","article-title":"Psychological Conditions of Personal Engagement and Disengagement at Work","volume":"33","author":"Kahn","year":"1990","journal-title":"Acad. Manag. J."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1016\/j.chb.2015.07.045","article-title":"Challenging games help students learn: An empirical study on engagement, flow and immersion in game-based learning","volume":"54","author":"Hamari","year":"2016","journal-title":"Comput. Hum. Behav."},{"key":"ref_61","unstructured":"Csikszentmihalyi, M. (1990). Flow: The Psychology of Optimal Experience, Harper & Row."},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1016\/j.entcom.2015.11.002","article-title":"Media enjoyment as a function of control over characters","volume":"12","author":"Rogers","year":"2016","journal-title":"Entertain. Comput."},{"key":"ref_63","doi-asserted-by":"crossref","unstructured":"McKee, A. (2016). FUN!: What Entertainment Tells Us about Living a Good Life, Springer.","DOI":"10.1057\/978-1-137-49179-4"},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"159","DOI":"10.4236\/psych.2010.13021","article-title":"\u201cFun, Fun, Fun\u201d: Types of Fun, Attitudes to Fun, and their Relation to Personality and Biographical Factors","volume":"1","author":"McManus","year":"2010","journal-title":"Psychology"},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1007\/s10111-007-0069-9","article-title":"Validating the Fun Toolkit: An instrument for measuring children\u2019s opinions of technology","volume":"10","author":"Read","year":"2008","journal-title":"Cogn. Technol. Work"},{"key":"ref_66","unstructured":"Van der Sluis, F., van Dijk, B., and Perloy, B. (2012, January 28\u201331). Measuring fun and enjoyment of children in a museum: Evaluating the Smileyometer. Proceedings of the Measuring Behavior 2012, Utrecht, The Netherlands."},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Tisza, G., and Markopoulos, P. FunQ: Measuring the fun experience of a learning activity with adolescents. Curr. Psychol., 2021.","DOI":"10.1007\/s12144-021-01484-2"},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"155","DOI":"10.5214\/ans.0972.7531.200408","article-title":"Memory: A contribution to experimental psychology","volume":"20","author":"Ebbinghaus","year":"2013","journal-title":"Ann. Neurosci."},{"key":"ref_69","doi-asserted-by":"crossref","unstructured":"Thorndike, L., and Bruce, D. (2017). Animal Intelligence: Experimental Studies, Routledge.","DOI":"10.4324\/9781351321044"},{"key":"ref_70","unstructured":"Hilgard, E.R. (1953). Introduction to Psychology, Harcourt Brace."},{"key":"ref_71","unstructured":"Roediger, H.L., Capaldi, E.D., Paris, S.G., Polivy, J., and Herman, C.P. (1995). Psychology, Wadsworth Publishing. [4th ed.]."},{"key":"ref_72","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1177\/0092055X14527909","article-title":"Measuring Student Learning in Social Statistics: A Pretest-Posttest Study of Knowledge Gain","volume":"42","author":"Delucchi","year":"2014","journal-title":"Teach. Sociol."},{"key":"ref_73","unstructured":"Campbell, J., McCloy, R., Oppler, S., and Sager, C. (1993). A Theory of Performance, Jossey-Bass."},{"key":"ref_74","unstructured":"Institute, C.F. (2022, October 28). Sales Revenue. Available online: https:\/\/corporatefinanceinstitute.com\/resources\/knowledge\/accounting\/sales-revenue\/."},{"key":"ref_75","doi-asserted-by":"crossref","unstructured":"Berkman, L.F., Kawachi, I., and Theorell, T. (2014). Social Epidemiology, Oxford University Press.","DOI":"10.1093\/med\/9780195377903.001.0001"},{"key":"ref_76","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1097\/00006842-199805000-00004","article-title":"Psychosocial Work Characteristics and Social Support as Predictors of SF-36 Health Functioning: The Whitehall II Study","volume":"60","author":"Stansfeld","year":"1998","journal-title":"Psychosom. Med."},{"key":"ref_77","unstructured":"World Health Organization (2022, October 28). Stress. Available online: https:\/\/www.who.int\/news-room\/questions-and-answers\/item\/stress."},{"key":"ref_78","unstructured":"Goldberg, R.C. (2022). Exploring the Relationships among Bio-Psycho-Social Measures of Stress: A Multifactorial Approach Towards the Evaluation and Reduction of Stress. [Ph.D. Thesis, California Institute of Integral Studies]."},{"key":"ref_79","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1016\/0022-3999(93)90120-5","article-title":"Development of the perceived stress questionnaire: A new tool for psychosomatic research","volume":"37","author":"Levenstein","year":"1993","journal-title":"J. Psychosom. Res."},{"key":"ref_80","unstructured":"(2011). Systems and Software Engineering\u2014Systems and Software Quality Requirements and Evaluation (SQuaRE)\u2014System and Software Quality Models (Standard No. ISO\/IEC 25010)."},{"key":"ref_81","unstructured":"Dama\u0161evi\u010dius, R., and Vasiljevien\u0117, G. (2019, January 10\u201312). Design and Implementation of Rule Execution Mechanism for an eGuide Gamification Web Service. Proceedings of the Information and Software Technologies, Vilnius, Lithuania."},{"key":"ref_82","doi-asserted-by":"crossref","first-page":"3605","DOI":"10.1109\/TFUZZ.2018.2838560","article-title":"Belief reliability for uncertain random systems","volume":"26","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Fuzzy Syst."},{"key":"ref_83","doi-asserted-by":"crossref","first-page":"517","DOI":"10.1016\/S0166-4972(99)00166-2","article-title":"On reliability and maintenance management of electronic equipment in the tropics","volume":"20","author":"Oyebisi","year":"2000","journal-title":"Technovation"},{"key":"ref_84","doi-asserted-by":"crossref","unstructured":"Haring, G., Lindemann, C., and Reiser, M. (2000). Performance Evaluation: Origins and Directions, Springer.","DOI":"10.1007\/3-540-46506-5"},{"key":"ref_85","doi-asserted-by":"crossref","unstructured":"Bondi, A.B. (2000, January 17\u201320). Characteristics of scalability and their impact on performance. Proceedings of the 2nd International Workshop on Software and Performance, Ottawa, OT, Canada.","DOI":"10.1145\/350391.350432"},{"key":"ref_86","unstructured":"Kenton, W. (2022, October 28). Rate of Adoption. Investopedia. Available online: https:\/\/www.investopedia.com\/terms\/r\/rate-of-adoption.asp."},{"key":"ref_87","unstructured":"Santhanamery, T., and Ramayah, T. (2018). Encyclopedia of Information Science and Technology, IGI Global. [4th ed.]."},{"key":"ref_88","doi-asserted-by":"crossref","first-page":"351","DOI":"10.2307\/3250921","article-title":"Understanding Information Systems Continuance: An Expectation-Confirmation Model","volume":"25","author":"Bhattacherjee","year":"2001","journal-title":"MIS Q."},{"key":"ref_89","doi-asserted-by":"crossref","first-page":"102315","DOI":"10.1016\/j.ijinfomgt.2021.102315","article-title":"Continuance intention of online technologies: A systematic literature review","volume":"58","author":"Yan","year":"2021","journal-title":"Int. J. Inf. Manag."},{"key":"ref_90","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1287\/isre.2.3.173","article-title":"Predicting User Intentions: Comparing the Technology Acceptance Model with the Theory of Planned Behavior","volume":"2","author":"Mathieson","year":"1991","journal-title":"Inf. Syst. Res."},{"key":"ref_91","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1080\/10864415.2016.1121762","article-title":"Do Customers Identify with Our Website? The Effects of Website Identification on Repeat Purchase Intention","volume":"20","author":"King","year":"2016","journal-title":"Int. J. Electron. Commer."},{"key":"ref_92","doi-asserted-by":"crossref","unstructured":"Csikszentmihalyi, M. (2000). Beyond Boredom and Anxiety, Jossey-Bass.","DOI":"10.1037\/10516-164"},{"key":"ref_93","unstructured":"Sherry, J., Greenberg, B., Lucas, K., and Lachlan, K. (2006). Video Game Uses and Gratifications as Predictors of Use and Game Preference, American Psychological Association."},{"key":"ref_94","unstructured":"Ijsselsteijn, W., Hoogen, W., Klimmt, C., de Kort, Y., Lindley, C., Mathiak, K., Poels, K., Ravaja, N., Turpeinen, M., and Vorderer, P. (2008, January 26\u201329). Measuring the Experience of Digital Game Enjoyment. Proceedings of the International Conference on Methods and Techniques in Behavioral Research, Maastricht, The Netherlands."},{"key":"ref_95","doi-asserted-by":"crossref","unstructured":"van den Herik, H.J., and Spronck, P. (2010). Advances in Computer Games, 12th International Conference, ACG 2009, Pamplona, Spain, 11\u201313 May 2010, Springer.","DOI":"10.1007\/978-3-642-12993-3"},{"key":"ref_96","unstructured":"Sirlin, D. (2022, October 28). Balancing Multiplayer Competitive Games. Game Developer\u2019s Conference. Available online: https:\/\/www.gdcvault.com\/play\/1570\/Balancing-Multiplayer-Competitive."},{"key":"ref_97","unstructured":"Newheiser, M. (Strange Horizons, 2009). Playing fair: A look at competition in gaming, Strange Horizons."},{"key":"ref_98","unstructured":"Zuo, W. (2016). Why Are You Satisfied with an Online Game? Exploring Game Attractiveness and Gaming Climate from a Socio-Technical Perspective. [Master\u2019s Thesis, Department of Information Processing Science, University of Oulu]."},{"key":"ref_99","unstructured":"Cairns, P., Cox, A., and Nordin, A.I. (2014). Handbook of Digital Games, John Wiley & Sons, Ltd."},{"key":"ref_100","unstructured":"Murray, J.H. (1997). Hamlet on the Holodeck: The Future of Narrative in Cyberspace, The Free Press."},{"key":"ref_101","doi-asserted-by":"crossref","unstructured":"Patrick, E., Cosgrove, D., Slavkovic, A., Rode, J.A., Verratti, T., and Chiselko, G. (2000, January 1\u20136). Using a Large Projection Screen as an Alternative to Head-Mounted Displays for Virtual Environments. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, The Hague, The Netherlands.","DOI":"10.1145\/332040.332479"},{"key":"ref_102","doi-asserted-by":"crossref","first-page":"641","DOI":"10.1016\/j.ijhcs.2008.04.004","article-title":"Measuring and defining the experience of immersion in games","volume":"66","author":"Jennett","year":"2008","journal-title":"Int. J. Hum. Comput. Stud."},{"key":"ref_103","unstructured":"Cox, A.L., Cairns, P., Berthouze, N., and Jennett, C. (2006). Proceedings of the CogSci 2006 Workshop: What Have Eye Movements Told Us So Far, and What Is Next, Cognitive Science Society."},{"key":"ref_104","unstructured":"Davis, F.D. (1985). A Technology Acceptance Model for Empirically Testing New End-User Information Systems: Theory and Results. [Ph.D. Thesis, Massachusetts Institute of Technology]."},{"key":"ref_105","unstructured":"Komischke, T. (2022, December 29). Usability Testing: 7 Metrics to Assess Ease of Use. CMSWire. Available online: https:\/\/www.cmswire.com\/digital-experience\/usability-testing-7-metrics-to-assess-ease-of-use\/."},{"key":"ref_106","doi-asserted-by":"crossref","unstructured":"MacDonald, C.M., and Atwood, M.E. (2014, January 21\u201325). What Does It Mean for a System to Be Useful? An Exploratory Study of Usefulness. Proceedings of the 2014 Conference on Designing Interactive Systems, Vancouver, BC, Canada.","DOI":"10.1145\/2598510.2598600"},{"key":"ref_107","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1016\/j.jbusres.2019.01.031","article-title":"Having fun while receiving rewards?: Exploration of gamification in loyalty programs for consumer loyalty","volume":"106","author":"Hwang","year":"2020","journal-title":"J. Bus. Res."},{"key":"ref_108","doi-asserted-by":"crossref","unstructured":"Barna, B., and Fodor, S. (2018, January 6\u20138). Gamification\u2019s Impact on Employee Engagement: Enhancing Employee Well-Being with a Cloud Based Gamified Team-Building Application. Proceedings of the 2018 6th International Conference on Future Internet of Things and Cloud Workshops (FiCloudW), Barcelona, Spain.","DOI":"10.1109\/W-FiCloud.2018.00039"},{"key":"ref_109","doi-asserted-by":"crossref","first-page":"260","DOI":"10.1111\/1748-8583.12272","article-title":"The effects of a gamified human resource management system on job satisfaction and engagement","volume":"30","author":"Silic","year":"2020","journal-title":"Hum. Resour. Manag. J."},{"key":"ref_110","doi-asserted-by":"crossref","unstructured":"Jantke, K.P., Krebs, J., and Santoso, M. (2014, January 7\u201310). Game Amusement & CRM: Castle Scharfenstein AR case study. Proceedings of the 2014 IEEE 3rd Global Conference on Consumer Electronics (GCCE), Tokyo, Japan.","DOI":"10.1109\/GCCE.2014.7031173"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/14\/2\/136\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:36:23Z","timestamp":1760121383000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/14\/2\/136"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,19]]},"references-count":110,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["info14020136"],"URL":"https:\/\/doi.org\/10.3390\/info14020136","relation":{},"ISSN":["2078-2489"],"issn-type":[{"type":"electronic","value":"2078-2489"}],"subject":[],"published":{"date-parts":[[2023,2,19]]}}}