{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:46:04Z","timestamp":1760060764700,"version":"build-2065373602"},"reference-count":38,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T00:00:00Z","timestamp":1759968000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computers"],"abstract":"<jats:p>Analyzing student performance in Introductory Programming courses in Higher Education is critical for early intervention and improved learning outcomes. This study explores the potential of a cognitive test for student success in an Introductory Programming course by analyzing data from 180 students, including Freshmen and Repeating Students, using descriptive statistics, correlation analysis, Categorical Principal Component Analysis and Item Response Theory models analysis. Analysis of the cognitive test revealed that some reasoning questions presented a statistically significant correlation, albeit of weak magnitude, with the course grades, particularly for freshman students. The development of models for predicting student performance in Introductory Programming using cognitive tests is also being explored. This study found that reasoning skills, namely logical reasoning and sequence completion, were more predictive of success in programming than general ability. The study also showed that a Programming Cognitive Test can be a useful tool for identifying students at risk of failure, particularly for freshmen students.<\/jats:p>","DOI":"10.3390\/computers14100428","type":"journal-article","created":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T09:41:02Z","timestamp":1760002862000},"page":"428","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Measure Student Aptitude in Learning Programming in Higher Education\u2014A Data Analysis"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-0058-1386","authenticated-orcid":false,"given":"Jo\u00e3o","family":"Pires","sequence":"first","affiliation":[{"name":"Institute of Engineering, Polytechnic University of Coimbra, Rua da Miseric\u00f3rdia, Lagar dos Corti\u00e7os, S. Martinho do Bispo, 3045-093 Coimbra, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3167-8714","authenticated-orcid":false,"given":"Ana Rosa","family":"Borges","sequence":"additional","affiliation":[{"name":"Institute of Engineering, Polytechnic University of Coimbra, Rua da Miseric\u00f3rdia, Lagar dos Corti\u00e7os, S. Martinho do Bispo, 3045-093 Coimbra, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9660-2011","authenticated-orcid":false,"given":"Jorge","family":"Bernardino","sequence":"additional","affiliation":[{"name":"Institute of Engineering, Polytechnic University of Coimbra, Rua da Miseric\u00f3rdia, Lagar dos Corti\u00e7os, S. Martinho do Bispo, 3045-093 Coimbra, Portugal"},{"name":"Centre for Informatics and Systems, University of Coimbra, Polo II, Pinhal de Marrocos, 3030-290 Coimbra, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8330-1903","authenticated-orcid":false,"given":"Fernanda","family":"Brito Correia","sequence":"additional","affiliation":[{"name":"Institute of Engineering, Polytechnic University of Coimbra, Rua da Miseric\u00f3rdia, Lagar dos Corti\u00e7os, S. Martinho do Bispo, 3045-093 Coimbra, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8418-8095","authenticated-orcid":false,"given":"Anabela","family":"Gomes","sequence":"additional","affiliation":[{"name":"Institute of Engineering, Polytechnic University of Coimbra, Rua da Miseric\u00f3rdia, Lagar dos Corti\u00e7os, S. Martinho do Bispo, 3045-093 Coimbra, Portugal"},{"name":"Centre for Informatics and Systems, University of Coimbra, Polo II, Pinhal de Marrocos, 3030-290 Coimbra, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2025,10,9]]},"reference":[{"key":"ref_1","unstructured":"Jenkins, T. (2002, January 27\u201329). On the Difficulty of Learning to Program. Proceedings of the 3rd Annual Conference of the LTSN Centre for Information and Computer Sciences, Loughborough, UK."},{"key":"ref_2","first-page":"1","article-title":"Students\u2019 misconceptions and other difficulties in introductory programming: A literature review","volume":"18","author":"Qian","year":"2017","journal-title":"ACM Trans. Comput. Educ. (TOCE)"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Bennedsen, J., Caspersen, M., and K\u00f6lling, M. (2008). Reflections on the Teaching of Programming, Springer.","DOI":"10.1007\/978-3-540-77934-6"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1145\/1151954.1067453","article-title":"A study of the difficulties of novice programmers","volume":"37","author":"Lahtinen","year":"2005","journal-title":"ACM SIGCSE Bull."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Watson, C., and Li, F.W.B. (2014, January 21\u201325). Failure rates in introductory programming revisited. Proceedings of the 2014 Innovation and Technology in Computer Science Education Conference\u2014ITiCSE 2014, Uppsala, Sweden.","DOI":"10.1145\/2591708.2591749"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1145\/3324888","article-title":"Failure rates in introductory programming\u201412 years later","volume":"10","author":"Bennedsen","year":"2019","journal-title":"ACM Inroads"},{"key":"ref_7","unstructured":"Gomes, A., and Mendes, A.J. (2007, January 3\u20137). Learning to program-difficulties and solutions. Proceedings of the International Conference on Engineering Education\u2014ICEE 2007, Coimbra, Portugal."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1145\/1138403.1138430","article-title":"Abstraction ability as an indicator of success for learning object-oriented programming?","volume":"38","author":"Bennedsen","year":"2006","journal-title":"ACM SIGCSE Bull."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1145\/507758.377467","article-title":"The effect of student attributes on success in programming","volume":"33","author":"Byrne","year":"2001","journal-title":"ACM SIGCSE Bull."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Luxton-Reilly, A., Simon Albluwi, I., Becker, B.A., Giannakos, M., Kumar, A.N., Ott, L., Paterson, J., Scott, M.J., Sheard, J., and Szabo, C. (2018, January 2\u20134). Introductory programming: A systematic literature review. Proceedings of the Annual Conference on Innovation and Technology in Computer Science Education\u2014ITiCSE-2018, Larnaca, Cyprus.","DOI":"10.1145\/3293881.3295779"},{"key":"ref_11","unstructured":"Gomes, A., and Mendes, A. (July, January 30). A study on student\u2019s characteristics and programming learning. Proceedings of the 2008 World Conference on Educational Multimedia, Hypermedia & Telecommunications, EDMEDIA 2008, Vienna, Austria."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Gomes, A., and Mendes, A. (2014, January 22\u201325). A teacher\u2019s view about introductory programming teaching and learning: Difficulties, strategies and motivations. Proceedings of the Frontiers in Education Conference\u2014FIE-2014, Madrid, Spain.","DOI":"10.1109\/FIE.2014.7044086"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Tomai, E., and Reilly, C.F. (2014, January 5\u20138). The impact of math preparedness on introductory programming (CS1) success (abstract only). Proceedings of the 45th ACM Technical Symposium on Computer Science Education, Atlanta, GA, USA.","DOI":"10.1145\/2538862.2544292"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Lishinski, A., Yadav, A., Enbody, R., and Good, J. (2016, January 2\u20135). The influence of problem solving abilities on students\u2019 performance on different assessment tasks in CS1. Proceedings of the 47th ACM Technical Symposium on Computing Science Education, Memphis, TN, USA.","DOI":"10.1145\/2839509.2844596"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Sobral, S.R. (2021). Strategies on Teaching Introducing to Programming in Higher Education. Trends and Applications in Information Systems and Technologies, Springer. Advances in Intelligent Systems and Computing.","DOI":"10.1007\/978-3-030-72660-7_14"},{"key":"ref_16","unstructured":"Tavares, P.C., Henriques, P.R., and Gomes, E.F. (2017, January 21\u201323). A computer platform to increase motivation in programming students-PEP. Proceedings of the 9th International Conference on Computer Supported Education\u2014CSEDU 2017, Porto, Portugal."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.compedu.2011.08.015","article-title":"A distributed system for learning programming on-line","volume":"58","author":"Regueras","year":"2012","journal-title":"Comput. Educ."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Ferreira, A., Gomes, A., and Mendes, A.J. (2022, January 17\u201319). SICAS2: Interactive Tool to Support Programming Learning. Proceedings of the 24th International Symposium on Computers in Education\u2014SIIE 2022, Coimbra, Portugal.","DOI":"10.1109\/SIIE56031.2022.9982323"},{"key":"ref_19","unstructured":"Moreno, A., Myller, N., and Sutinen, E. (2004, January 26\u201329). JeCo, a collaborative learning tool for programming. Proceedings of the 2004 IEEE Symposium on Visual Languages and Human Centric Computing, Rome, Italy."},{"key":"ref_20","first-page":"1","article-title":"Learning to program with personal robots: Influences on student motivation","volume":"12","author":"McGill","year":"2012","journal-title":"ACM Trans. Comput. Educ."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"764","DOI":"10.1037\/edu0000314","article-title":"The cognitive benefits of learning computer programming: A meta-analysis of transfer effects","volume":"111","author":"Scherer","year":"2019","journal-title":"J. Educ. Psychol."},{"key":"ref_22","first-page":"149","article-title":"Testing Programming Aptitude in Introductory Programming Courses","volume":"30","author":"Harris","year":"2014","journal-title":"J. Comput. Sci. Coll."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Hicks, E., and Phan, V. (March, January 26). In-class Coding Exercises As A Mechanism To Inform Early Intervention In Programming Courses. Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE 2025), Pittsburgh, PA, USA.","DOI":"10.1145\/3641554.3701802"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Gao, G., Marwan, S., and Price, T.W. (2021). Early Performance Prediction using Interpretable Patterns in Programming Process Data. arXiv.","DOI":"10.1145\/3408877.3432439"},{"key":"ref_25","unstructured":"Schantong, B., Gorgosch, D., and Siegmund, J. (2024). Toward Finding and Supporting Struggling Students in a Programming Course with an Early Warning System. arXiv."},{"key":"ref_26","first-page":"121","article-title":"Predictors of success in a first programming course","volume":"34","author":"Simon","year":"2006","journal-title":"ACM SIGCSE Bull."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Quille, K., Nam Liao, S., Costelloe, E., Nolan, K., Mooney, A., and Shah, K. (2022, January 8\u201313). PreSS: Predicting Student Success Early in CS1. A Pilot International Replication and Generalization Study. Proceedings of the Annual Conference on Innovation and Technology in Computer Science Education, ITiCSE-2022, Dublin, Ireland.","DOI":"10.1145\/3502718.3524755"},{"key":"ref_28","first-page":"14","article-title":"The programming performance prophecies: Predicting student achievement in a first-year introductory programming course","volume":"22","author":"Ringenberg","year":"2012","journal-title":"Comput. Educ. J."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1145\/358808.358811","article-title":"Identifying potential to acquire programming skill","volume":"23","author":"Mazlack","year":"1980","journal-title":"Commun. ACM"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Smith, C.E., Shiekh, K., Cooreman, H., Rahman, S., Zhu, Y., Siam, M.K., Ivanitskiy, M., Ahmed, A.M., Hallinan, M., and Grisak, A. (2024, January 8\u201310). Early Adoption of Generative Artificial Intelligence in Computing Education: Emergent Student Use Cases and Perspectives in 2023. Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1 (ITiCSE 2024), Milan, Italy.","DOI":"10.1145\/3649217.3653575"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"100183","DOI":"10.1016\/j.caeo.2024.100183","article-title":"Enhancing active learning through collaboration between human teachers and generative AI","volume":"6","author":"Pahi","year":"2024","journal-title":"Comput. Educ. Open"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Li, N., Broner, S., Kim, Y., Mizuo, K., Sauder, E., To, C., Wang, A., Gila, O., and Shindler, M. (March, January 26). Investigating the Capabilities of Generative AI in Solving Data Structures, Algorithms, and Computability Problems. Proceedings of the 56th ACM Technical Symposium on Computer Science Education V. 1 (SIGCSETS 2025), Pittsburgh, PA, USA.","DOI":"10.1145\/3641554.3701946"},{"key":"ref_33","unstructured":"(2024, April 01). IBM Verbal Questions|Verbal Ability Questions For IBM. Available online: https:\/\/cpt.hitbullseye.com\/IBM-Verbal-Questions.php."},{"key":"ref_34","unstructured":"(2024, April 01). IBM Reasoning Questions|Reasoning Test For IBM. Available online: https:\/\/cpt.hitbullseye.com\/IBM-Reasoning-Test.php."},{"key":"ref_35","unstructured":"(2024, April 01). IBM Aptitude Questions|Aptitude Test For IBM. Available online: https:\/\/cpt.hitbullseye.com\/IBM-Aptitude-Questions.php."},{"key":"ref_36","unstructured":"Ragni, M., Kola, K., and Johnson-Laird, P.N. (2017, January 16\u201329). The Wason Selection Task: A Meta-Analysis. Proceedings of the 39th Annual Meeting of the Cognitive Science Society: Computational Foundations of Cognition\u2014CogSci 2017, London, UK."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1111\/j.2044-8295.1996.tb02587.x","article-title":"Deciding before you think: Relevance and reasoning in the selection task","volume":"87","author":"Evans","year":"1996","journal-title":"Br. J. Psychol."},{"key":"ref_38","first-page":"103","article-title":"Applications, reliability and validity of the index of learning styles","volume":"21","author":"Felder","year":"2005","journal-title":"Int. J. Eng. Educ."}],"container-title":["Computers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-431X\/14\/10\/428\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:50:50Z","timestamp":1760035850000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-431X\/14\/10\/428"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,9]]},"references-count":38,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2025,10]]}},"alternative-id":["computers14100428"],"URL":"https:\/\/doi.org\/10.3390\/computers14100428","relation":{},"ISSN":["2073-431X"],"issn-type":[{"type":"electronic","value":"2073-431X"}],"subject":[],"published":{"date-parts":[[2025,10,9]]}}}