{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,10]],"date-time":"2026-02-10T18:24:37Z","timestamp":1770747877439,"version":"3.49.0"},"reference-count":22,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2019,2,7]],"date-time":"2019-02-07T00:00:00Z","timestamp":1549497600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Solving programming exercises is the best way to promote practice in computer programming courses and, hence, to learn a programming language. Meanwhile, programming courses continue to have an high rate of failures and dropouts. The main reasons are related with the inherent domain complexity, the teaching methodologies, and the absence of automatic systems with features such as intelligent authoring, profile-based exercise sequencing, content adaptation, and automatic evaluation on the student\u2019s resolution. At the same time, gamification is being used as an approach to engage learners\u2019 motivations. Despite its success, its implementation is still complex and based on ad-hoc and proprietary solutions. This paper presents PROud as a framework to inject gamification features in computer programming learning environments based on the usage data from programming exercises. This data can be divided into two categories: generic data produced by the learning environment\u2014such as, the number of attempts and the duration that the students took to solve a specific exercise\u2014or code-specific data produced by the assessment tool\u2014such as, code size, use memory, or keyword detection. The data is gathered in cloud storage and can be consumed by the learning environment through the use of a client library that communicates with the server through an established Application Programming Interface (API). With the fetched data, the learning environment can generate new gamification assets (e.g., leaderboards, quests, levels) or enrich content adaptations and recommendations in the inner components such as the sequencing tools. The framework is evaluated on its usefulness in the creation of a gamification asset to present dynamic statistics on specific exercises.<\/jats:p>","DOI":"10.3390\/info10020054","type":"journal-article","created":{"date-parts":[[2019,2,7]],"date-time":"2019-02-07T11:50:33Z","timestamp":1549540233000},"page":"54","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["PROud\u2014A Gamification Framework Based on Programming Exercises Usage Data"],"prefix":"10.3390","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1985-6285","authenticated-orcid":false,"given":"Ricardo","family":"Queir\u00f3s","sequence":"first","affiliation":[{"name":"Department of Informatics, School of Media Arts and Design (ESMAD), Polytechnic of Porto, 4480-786 Vila do Conde, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2019,2,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1080\/08993400500150747","article-title":"A Survey of Automated Assessment Approaches for Programming Assignments","volume":"15","year":"2005","journal-title":"Comput. Sci. Educ."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1145\/1140123.1140182","article-title":"RoboCode & Problem-based Learning: A Non-prescriptive Approach to Teaching Programming","volume":"38","author":"Gibson","year":"2006","journal-title":"SIGCSE Bull."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1076\/csed.13.2.137.14200","article-title":"Learning and Teaching Programming: A Review and Discussion","volume":"13","author":"Robins","year":"2003","journal-title":"Comput. Sci. Educ."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.compedu.2011.08.015","article-title":"A Distributed System for Learning Programming On-line","volume":"58","author":"Regueras","year":"2012","journal-title":"Comput. Educ."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Coelho, A., Kato, E., Xavier, J.A., and Gon\u00e7alves, R. (2011, January 19\u201320). Serious Game for Introductory Programming. Proceedings of the Second International Conference on Serious Games Development and Applications (SGDA\u201911), Lisbon, Portugal.","DOI":"10.1007\/978-3-642-23834-5_6"},{"key":"ref_6","unstructured":"De Queir\u00f3s, R.A.P. (2019). Code Generation, Analysis Tools, and Testing for Quality, IGI-Global."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"De Queir\u00f3s, R.A.P. (2019). A Survey on Computer Programming Learning Environments. Code Generation, Analysis Tools, and Testing for Quality, IGI-Global. Chapter 4.","DOI":"10.4018\/978-1-5225-7455-2.ch004"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Von Hausswolff, K. (2017, January 18\u201320). Hands-on in Computer Programming Education. Proceedings of the 2017 ACM Conference on International Computing Education Research (ICER \u201917), Tacoma, WA, USA.","DOI":"10.1145\/3105726.3105735"},{"key":"ref_9","unstructured":"Ortiz, M., Chiluiza, K., and Valcke, M. (2017, January 5\u20136). Gamification in Computer Programming: Effects on learning, engagement, self-efficacy and intrinsic motivation. Proceedings of the European Conference on Game Based Learning, Graz, Austria."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Swacha, J., and Muszynska, K. (2016, January 2\u20134). Design patterns for gamification of work. Proceedings of the Fourth International Conference on Technological Ecosystems for Enhancing Multiculturality, Salamanca, Spain.","DOI":"10.1145\/3012430.3012604"},{"key":"ref_11","first-page":"160","article-title":"Learner participation and engagement in open online courses: Insights from the Peer 2 Peer University","volume":"9","author":"Ahn","year":"2013","journal-title":"MERLOT J. Online Learn. Teach."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Layth Khaleel, F., Ashaari, N., Wook, T., Tengku Wook, T.S.M.T.W., and Ismail, A. (2015, January 8\u201310). The study of gamification application architecture for programming language course. Proceedings of the 9th International Conference on Ubiquitous Information Management and Communication, Bali, Indonesia.","DOI":"10.1145\/2701126.2701222"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Queir\u00f3s, R., Leal, J.P., and Paiva, J.C. (2016). Integrating rich learning applications in LMS. State-of-the-Art and Future Directions of Smart Learning, Springer.","DOI":"10.1007\/978-981-287-868-7_46"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1109\/MIC.2007.70","article-title":"Service-oriented e-learning platforms: From monolithic systems to flexible services","volume":"11","author":"Dagger","year":"2007","journal-title":"IEEE Internet Comput."},{"key":"ref_15","unstructured":"Leal, J.P., and Queir\u00f3s, R. (2019, February 06). Using the Learning Tools Interoperability Framework for LMS Integration in Service Oriented Architectures 2011. Available online: http:\/\/recipp.ipp.pt\/handle\/10400.22\/4724."},{"key":"ref_16","unstructured":"(2019, February 06). LMS Market by Component (Solution and Services), Delivery Mode (Distance Learning, Instructor-Led Training and Blended Learning), Deployment Type, User Type (Academic and Corporate), and Region - Global Forecast to 2023. Technical Report, 2018. Available online: https:\/\/www.marketsandmarkets.com\/Market-Reports\/learning-management-systems-market-1266.html."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Leal, J.P., and Queir\u00f3s, R. (2012). A comparative study on LMS interoperability. Higher Education Institutions and Learning Management Systems: Adoption and Standardization, IGI Global.","DOI":"10.4018\/978-1-60960-884-2.ch007"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1007\/978-3-642-01347-8_9","article-title":"CrimsonHex: A Service Oriented Repository of Specialised Learning Objects","volume":"Volume 24","author":"Filipe","year":"2009","journal-title":"Enterprise Information Systems"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"567","DOI":"10.1002\/spe.522","article-title":"Mooshak: A Web-based multi-site programming contest system","volume":"33","author":"Leal","year":"2003","journal-title":"Softw. Pract. Exp."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Al-Smadi, M., and G\u00fctl, C. (2010, January 14). SOA-based Architecture for a Generic and Flexible E-assessment System. Proceedings of the IEEE Conference on Education Engineering (EDUCON), Madrid, Spain.","DOI":"10.1109\/EDUCON.2010.5492537"},{"key":"ref_21","first-page":"1454","article-title":"Orchestration of E-Learning Services for Automatic Evaluation of Programming Exercises","volume":"18","author":"Leal","year":"2012","journal-title":"J. Univers. Comput. Sci."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1479","DOI":"10.2298\/CSIS131005074Q","article-title":"Sequencing educational resources with Seqins","volume":"11","author":"Leal","year":"2014","journal-title":"Comput. Sci. Inf. Syst."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/10\/2\/54\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:31:32Z","timestamp":1760185892000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/10\/2\/54"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,2,7]]},"references-count":22,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2019,2]]}},"alternative-id":["info10020054"],"URL":"https:\/\/doi.org\/10.3390\/info10020054","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,2,7]]}}}