{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T06:20:13Z","timestamp":1791526813233,"version":"4.3.4"},"reference-count":38,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2024,9,18]],"date-time":"2024-09-18T00:00:00Z","timestamp":1726617600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,9,18]],"date-time":"2024-09-18T00:00:00Z","timestamp":1726617600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003759","name":"Universidad Polit\u00e9cnica de Madrid","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100003759","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Educ Inf Technol"],"published-print":{"date-parts":[[2025,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Classroom response systems (CRS) represent an innovative educational technology that can be used to promote active learning and student engagement. This study explores the effectiveness of CRS in enhancing student learning and performance across various engineering courses related to heat engines. During five academic years, CRS have been used weekly in six subjects that belong to two bachelor\u2019s degrees and one master\u2019s degree to promote continuous learning. The performance of the students in the CRS benefits the student by up to\u2009+\u200910% in the final mark. In this period, data from more than 1,000 students has been collected, including key performance indicators (KPIs) derived from <jats:italic>Kahoot!<\/jats:italic>, such as the number and correctness of responses and time spent answering, and other evaluations such as theory and numerical problem exams results. The objective of the study is to find the KPI that best identifies the knowledge acquisition by the students. In order to do so, the correlation of the KPI with the final exams is analysed by means of the correlation of determination and the root mean square error. The findings reveal that KPIs based on the number of questions asked exhibit stronger correlations with exam results compared to those based on responses or correctness. The study also highlights the positive impact of CRS on student engagement, classroom dynamics, and exam preparation, supported by student feedback. In essence, this research provides insights into the relationship between CRS metrics and academic outcomes, offering valuable considerations for the integration of technology in engineering education.<\/jats:p>","DOI":"10.1007\/s10639-024-13048-5","type":"journal-article","created":{"date-parts":[[2024,9,18]],"date-time":"2024-09-18T18:02:19Z","timestamp":1726682539000},"page":"861-879","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Evaluating classroom response systems in engineering education: Which metrics better reflect student performance?"],"prefix":"10.1007","volume":"30","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3948-3292","authenticated-orcid":false,"given":"Rub\u00e9n","family":"Abbas","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2667-5909","authenticated-orcid":false,"given":"Andr\u00e9s","family":"Sebasti\u00e1n","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1193-3204","authenticated-orcid":false,"given":"Jes\u00fas","family":"Casanova","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,9,18]]},"reference":[{"key":"13048_CR1","unstructured":"Afreen, R. (2014). Bring your own device (BYOD) in higher education: Opportunities and challenges. International Journal of Emerging Trends & Technology in Computer Science, 3(1), 233\u2013236. http:\/\/www.ijettcs.org\/Volume3Issue1\/IJETTCS-2014-02-25-117.pdf"},{"key":"13048_CR2","doi-asserted-by":"publisher","unstructured":"Albero, V., & Ib\u00e1\u00f1ez, C. (2018). Content review using Kahoot! with aeronautical engineering students. In 4th International Conference on Higher Education Advances (HEAD'18) (pp. 1117\u20131124). Editorial Universitat Polit\u00e8cnica de Val\u00e8ncia. https:\/\/doi.org\/10.4995\/HEAD18.2018.8158","DOI":"10.4995\/HEAD18.2018.8158"},{"issue":"4","key":"13048_CR3","doi-asserted-by":"publisher","first-page":"313","DOI":"10.1504\/IJLT.2015.074073","volume":"10","author":"A Aljaloud","year":"2015","unstructured":"Aljaloud, A., Gromik, N., Billingsley, W., & Kwan, P. (2015). Research trends in student response systems: A literature review. International Journal of Learning Technology, 10(4), 313\u2013325. https:\/\/doi.org\/10.1504\/IJLT.2015.074073","journal-title":"International Journal of Learning Technology"},{"issue":"22","key":"13048_CR4","doi-asserted-by":"publisher","first-page":"14957","DOI":"10.3390\/su142214957","volume":"14","author":"O Altwijri","year":"2022","unstructured":"Altwijri, O., Alsadoon, E., Shahba, A. A. W., Soufan, W., & Alkathiri, S. (2022). The Effect of Using \u201cStudent Response Systems (SRS)\u201d on Faculty Performance and Student Interaction in the Classroom. Sustainability, 14(22), 14957. https:\/\/doi.org\/10.3390\/su142214957","journal-title":"Sustainability"},{"key":"13048_CR5","doi-asserted-by":"publisher","unstructured":"Basuki, Y., & Hidayati, Y. (2019). Kahoot! or Quizizz: The students\u2019 perspectives. In Proceedings of the 3rd English Language and Literature International Conference (ELLiC) (pp. 202\u2013211). https:\/\/doi.org\/10.4108\/eai.27-4-2019.2285331","DOI":"10.4108\/eai.27-4-2019.2285331"},{"key":"13048_CR6","doi-asserted-by":"publisher","unstructured":"Batsila, M., & Tsihouridis, C. (2018). \u201cLet\u2019s go\u2026 Kahooting\u201d\u2013teachers\u2019 views on CRS for teaching purposes. In Teaching and Learning in a Digital World: Proceedings of the 20th International Conference on Interactive Collaborative Learning\u2013Volume 1 (pp. 563\u2013571). Springer International Publishing. https:\/\/doi.org\/10.1007\/978-3-319-73210-7_66","DOI":"10.1007\/978-3-319-73210-7_66"},{"issue":"1","key":"13048_CR7","doi-asserted-by":"publisher","first-page":"31","DOI":"10.1119\/1.2121753","volume":"74","author":"ID Beatty","year":"2006","unstructured":"Beatty, I. D., Gerace, W. J., Leonard, W. J., & Dufresne, R. J. (2006). Designing effective questions for classroom response system teaching. American Journal of Physics, 74(1), 31\u201339. https:\/\/doi.org\/10.1119\/1.2121753","journal-title":"American Journal of Physics"},{"key":"13048_CR8","doi-asserted-by":"publisher","DOI":"10.1353\/rhe.0.0124","author":"D Bruff","year":"2009","unstructured":"Bruff, D. (2009). Teaching with classroom response systems: Creating active learning environments. John Wiley & Sons. https:\/\/doi.org\/10.1353\/rhe.0.0124","journal-title":"John Wiley & Sons"},{"key":"13048_CR9","unstructured":"Caron, P. L., & Gely, R. (2004). Taking back the law school classroom: Using technology to foster active student learning. J. LegaL educ., 54, 551. Available at SSRN: https:\/\/ssrn.com\/abstract=527522"},{"key":"13048_CR10","doi-asserted-by":"publisher","unstructured":"Casanova, J. (1971). An instructional experiment in organic chemistry. The use of a student response system. Journal of Chemical Education, 48(7), 453. https:\/\/doi.org\/10.1021\/ed048p453","DOI":"10.1021\/ed048p453"},{"issue":"2","key":"13048_CR11","doi-asserted-by":"publisher","first-page":"486","DOI":"10.3926\/jotse.1269","volume":"11","author":"V Chernov","year":"2021","unstructured":"Chernov, V., Klas, S., & Furman Shaharabani, Y. (2021). Incorporating Kahoot! in core engineering courses: Student engagement and performance. Journal of Technology and Science Education, 11(2), 486\u2013497. https:\/\/doi.org\/10.3926\/jotse.1269","journal-title":"Journal of Technology and Science Education"},{"key":"13048_CR12","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1007\/s10956-006-0360-1","volume":"15","author":"C Fies","year":"2006","unstructured":"Fies, C., & Marshall, J. (2006). Classroom response systems: A review of the literature. Journal of Science Education and Technology, 15, 101\u2013109. https:\/\/doi.org\/10.1007\/s10956-006-0360-1","journal-title":"Journal of Science Education and Technology"},{"issue":"4","key":"13048_CR13","doi-asserted-by":"publisher","first-page":"1161","DOI":"10.1002\/cae.22512","volume":"30","author":"E Garcia-Lopez","year":"2022","unstructured":"Garcia-Lopez, E., & Garcia-Cabot, A. (2022). Implications of using classroom response systems (CRS) on learning performance: An experience of learning analytics. Computer Applications in Engineering Education, 30(4), 1161\u20131174. https:\/\/doi.org\/10.1002\/cae.22512","journal-title":"Computer Applications in Engineering Education"},{"key":"13048_CR14","doi-asserted-by":"publisher","unstructured":"Gumaelius, L., Skogh, I. B., Matth\u00edasd\u00f3ttir, \u00c1., & Pantzos, P. (2023). Engineering education in change. A case study on the impact of digital transformation on content and teaching methods in different engineering disciplines. European Journal of Engineering Education, 1\u201324. https:\/\/doi.org\/10.1080\/03043797.2023.2285794.","DOI":"10.1080\/03043797.2023.2285794"},{"key":"13048_CR15","unstructured":"Hanoa, E., (2023). 10 billion participants in 10 years!. Kahoot! Blogs.\u00a0https:\/\/kahoot.com\/blog\/2023\/11\/09\/kahoot-reaches-10-billion\/.\u00a0Accessed 17\/09\/2024."},{"key":"13048_CR16","doi-asserted-by":"publisher","first-page":"230","DOI":"10.1186\/s12909-019-1658-z","volume":"19","author":"MAA Ismail","year":"2019","unstructured":"Ismail, M. A. A., Ahmad, A., Mohammad, J. A. M., et al. (2019). Using Kahoot! as a formative assessment tool in medical education: A phenomenological study. BMC Medical Education, 19, 230. https:\/\/doi.org\/10.1186\/s12909-019-1658-z","journal-title":"BMC Medical Education"},{"key":"13048_CR17","unstructured":"Judson, E., & Sawada, D. (2002). Learning from past and present: Electronic response systems in college lecture halls. Journal of Computers in Mathematics and Science Teaching, 21(2), 167\u2013181. https:\/\/www.learntechlib.org\/primary\/p\/15113\/."},{"key":"13048_CR18","unstructured":"Kahoot! (2024a). online. https:\/\/kahoot.com\/. Accessed 13\/01\/2024."},{"key":"13048_CR19","unstructured":"Kahoot! (2024b). How Points Work, online. https:\/\/support.kahoot.com\/hc\/en-us\/articles\/115002303908-How-points-work. Accessed 01\/08\/2024."},{"key":"13048_CR20","doi-asserted-by":"publisher","unstructured":"Krath, J., Sch\u00fcrmann, L., von Korflesch, H. F. O. (2021), Revealing the theoretical basis of gamification: A systematic review and analysis of theory in research on gamification, serious games and game-based learning, Computers in Human Behavior, Volume 125, 106963. https:\/\/doi.org\/10.1016\/j.chb.2021.106963","DOI":"10.1016\/j.chb.2021.106963"},{"key":"13048_CR21","doi-asserted-by":"publisher","unstructured":"Lashari, T. A., Fiayaz, R., Lashari, S. A., Khan, I., Sultana, S., & Afzal, T. (2023) Kahoot: A game\u2010based web tool to assess motivation, engagement fun, and learning outcomes among engineers. Computer Applications in Engineering Education, e22684. https:\/\/doi.org\/10.1002\/cae.22684","DOI":"10.1002\/cae.22684"},{"issue":"2","key":"13048_CR22","doi-asserted-by":"publisher","first-page":"458","DOI":"10.1002\/cae.22465","volume":"30","author":"A Leon","year":"2022","unstructured":"Leon, A., & Pe\u00f1a, M. (2022). Gamification tools in the learning of shipbuilding in the undergraduate marine engineering education. Computer Applications in Engineering Education, 30(2), 458\u2013471. https:\/\/doi.org\/10.1002\/cae.22465","journal-title":"Computer Applications in Engineering Education"},{"issue":"1","key":"13048_CR23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s41039-018-0078-8","volume":"13","author":"SA Licorish","year":"2018","unstructured":"Licorish, S. A., Owen, H. E., Daniel, B., & George, J. L. (2018). Students\u2019 perception of Kahoot!\u2019s influence on teaching and learning. Research and Practice in Technology Enhanced Learning, 13(1), 1\u201323. https:\/\/doi.org\/10.1186\/s41039-018-0078-8","journal-title":"Research and Practice in Technology Enhanced Learning"},{"issue":"6","key":"13048_CR24","doi-asserted-by":"publisher","first-page":"652","DOI":"10.1080\/03043797.2012.719000","volume":"38","author":"I Lopez Arteaga","year":"2013","unstructured":"Lopez Arteaga, I., & Vinken, E. (2013). Example of good practice of a learning environment with a classroom response system in a mechanical engineering bachelor course. European Journal of Engineering Education, 38(6), 652\u2013660. https:\/\/doi.org\/10.1080\/03043797.2012.719000","journal-title":"European Journal of Engineering Education"},{"key":"13048_CR25","doi-asserted-by":"publisher","first-page":"13210","DOI":"10.3390\/ijerph182413210","volume":"18","author":"JJ L\u00f3pez-Jim\u00e9nez","year":"2021","unstructured":"L\u00f3pez-Jim\u00e9nez, J. J., Fern\u00e1ndez-Alem\u00e1n, J. L., Garc\u00eda-Bern\u00e1, J. A., L\u00f3pez Gonz\u00e1lez, L., Gonz\u00e1lez Sequeros, O., Nicol\u00e1s Ros, J., Carrillo de Gea, J. M., Idri, A., & Toval, A. (2021). Effects of Gamification on the Benefits of Student Response Systems in Learning of Human Anatomy: Three Experimental Studies. International Journal of Environmental Research and Public Health, 18, 13210. https:\/\/doi.org\/10.3390\/ijerph182413210","journal-title":"International Journal of Environmental Research and Public Health"},{"issue":"2","key":"13048_CR26","doi-asserted-by":"publisher","first-page":"732","DOI":"10.1016\/j.compedu.2012.03.011","volume":"59","author":"MC Mora","year":"2012","unstructured":"Mora, M. C., Sancho-Bru, J. L., Iserte, J. L., & S\u00e1nchez, F. T. (2012). An e-assessment approach for evaluation in engineering overcrowded groups. Computers & Education, 59(2), 732\u2013740. https:\/\/doi.org\/10.1016\/j.compedu.2012.03.011","journal-title":"Computers & Education"},{"key":"13048_CR27","doi-asserted-by":"publisher","first-page":"18989","DOI":"10.3402\/rlt.v21i0.18989","volume":"21","author":"KL Nielsen","year":"2013","unstructured":"Nielsen, K. L., Hansen, G., & Stav, J. B. (2013). Teaching with student response systems (SRS): Teacher-centric aspects that can negatively affect students\u2019 experience of using SRS. Research in Learning Technology, 21, 18989. https:\/\/doi.org\/10.3402\/rlt.v21i0.18989","journal-title":"Research in Learning Technology"},{"issue":"6","key":"13048_CR28","doi-asserted-by":"publisher","first-page":"1092","DOI":"10.1080\/03043797.2021.1997922","volume":"46","author":"C Paciarotti","year":"2021","unstructured":"Paciarotti, C., Bertozzi, G., & Sillaots, M. (2021). A new approach to Gamification in engineering education: The Learner-Designer Approach to Serious Games. European Journal of Engineering Education, 46(6), 1092\u20131116. https:\/\/doi.org\/10.1080\/03043797.2021.1997922","journal-title":"European Journal of Engineering Education"},{"key":"13048_CR29","doi-asserted-by":"publisher","unstructured":"\u015eahin, M. (2020). Classroom Response Systems as a Formative Assessment Tool: Investigation into Students\u2019 Perceived Usefulness and Behavioural Intention. International Journal of Assessment Tools in Education. 6(4):693\u2013705. https:\/\/doi.org\/10.21449\/ijate.576249","DOI":"10.21449\/ijate.576249"},{"key":"13048_CR30","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1007\/s10758-020-09487-x","volume":"27","author":"AN Saleem","year":"2022","unstructured":"Saleem, A. N., Noori, N. M., & Ozdamli, F. (2022). Gamification Applications in E-learning: A Literature Review. Tech Know Learn, 27, 139\u2013159. https:\/\/doi.org\/10.1007\/s10758-020-09487-x","journal-title":"Tech Know Learn"},{"issue":"6","key":"13048_CR31","doi-asserted-by":"publisher","first-page":"1872","DOI":"10.1002\/cae.22562","volume":"30","author":"SS Shareef","year":"2022","unstructured":"Shareef, S. S., & Rauf, H. L. (2022). Learning technical courses in architectural education through gamification. Computer Applications in Engineering Education, 30(6), 1872\u20131884. https:\/\/doi.org\/10.1002\/cae.22562","journal-title":"Computer Applications in Engineering Education"},{"issue":"4","key":"13048_CR32","doi-asserted-by":"publisher","first-page":"585","DOI":"10.1080\/03043797.2016.1209463","volume":"43","author":"P Shekhar","year":"2018","unstructured":"Shekhar, P., & Borrego, M. (2018). \u2018Not hard to sway\u2019: A case study of student engagement in two large engineering classes. European Journal of Engineering Education, 43(4), 585\u2013596. https:\/\/doi.org\/10.1080\/03043797.2016.1209463","journal-title":"European Journal of Engineering Education"},{"key":"13048_CR33","doi-asserted-by":"publisher","first-page":"1","DOI":"10.4236\/oalib.1109829","volume":"10","author":"WH Shi","year":"2023","unstructured":"Shi, W. H., & Hargis, J. (2023). Bridge the Gap\u2014Incorporating Classroom Response Systems for Classroom-Embedded Formative Assessment. Open Access Library Journal, 10, 1\u201319. https:\/\/doi.org\/10.4236\/oalib.1109829","journal-title":"Open Access Library Journal"},{"issue":"3","key":"13048_CR34","doi-asserted-by":"publisher","first-page":"398","DOI":"10.1109\/TE.2006.879802","volume":"49","author":"K Siau","year":"2006","unstructured":"Siau, K., Sheng, H., & Nah, F. H. (2006). Use of a classroom response system to enhance classroom interactivity. IEEE Transactions on Education, 49(3), 398\u2013403. https:\/\/doi.org\/10.1109\/TE.2006.879802","journal-title":"IEEE Transactions on Education"},{"key":"13048_CR35","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1016\/j.compedu.2014.11.004","volume":"82","author":"AI Wang","year":"2015","unstructured":"Wang, A. I. (2015). The wear out effect of a game-based student response system. Computers & Education, 82, 217\u2013227. https:\/\/doi.org\/10.1016\/j.compedu.2014.11.004","journal-title":"Computers & Education"},{"key":"13048_CR36","doi-asserted-by":"publisher","first-page":"103818","DOI":"10.1016\/j.compedu.2020.103818","volume":"149","author":"AI Wang","year":"2020","unstructured":"Wang, A. I., & Tahir, R. (2020). The effect of using Kahoot! for learning\u2013a literature review. Computers & Education, 149, 103818. https:\/\/doi.org\/10.1016\/j.compedu.2020.103818","journal-title":"Computers & Education"},{"key":"13048_CR37","doi-asserted-by":"publisher","unstructured":"Wang, A. I., \u00d8fsdal, T., & M\u00f8rch-Storstein, O. K. (2007). Lecture quiz - a mobile game concept for lectures. In IASTED international conference on software engineering and application (SEA 2007) (p. 6). Cambridge, MA, USA: Acta Press. https:\/\/doi.org\/10.5555\/1647636.1647688","DOI":"10.5555\/1647636.1647688"},{"issue":"4","key":"13048_CR38","doi-asserted-by":"publisher","first-page":"4507","DOI":"10.1007\/s10639-021-10459-6","volume":"26","author":"Q Zhang","year":"2021","unstructured":"Zhang, Q., & Yu, Z. (2021). A literature review on the influence of Kahoot! On learning outcomes, interaction, and collaboration. Education and Information Technologies, 26(4), 4507\u20134535. https:\/\/doi.org\/10.1007\/s10639-021-10459-6","journal-title":"Education and Information Technologies"}],"container-title":["Education and Information Technologies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10639-024-13048-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10639-024-13048-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10639-024-13048-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,17]],"date-time":"2025-01-17T08:54:52Z","timestamp":1737104092000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10639-024-13048-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,18]]},"references-count":38,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,1]]}},"alternative-id":["13048"],"URL":"https:\/\/doi.org\/10.1007\/s10639-024-13048-5","relation":{},"ISSN":["1360-2357","1573-7608"],"issn-type":[{"value":"1360-2357","type":"print"},{"value":"1573-7608","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,18]]},"assertion":[{"value":"19 February 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 September 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 September 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"None of the authors have a conflict of interest to disclose.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}]}}