{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T12:45:13Z","timestamp":1780663513037,"version":"3.54.1"},"reference-count":39,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2025,4,29]],"date-time":"2025-04-29T00:00:00Z","timestamp":1745884800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>As programming education becomes increasingly complex, grading student code has become a challenging task. Traditional methods, such as dynamic and static analysis, offer foundational approaches but often fail to provide granular insights, leading to inconsistencies in grading and feedback. This study addresses the limitations of these methods by integrating contrastive learning with explainable AI techniques to assess SQL code submissions. We employed contrastive learning to differentiate between student and correct SQL solutions, projecting them into a high-dimensional latent space, and used the Frobenius norm to measure the distance between these representations. This distance was used to predict the percentage of points deducted from each student\u2019s solution. To enhance interpretability, we implemented feature ablation and integrated gradients, which provide insights into the specific tokens in student code that impact the grading outcomes. Our findings indicate that this approach improves the accuracy, consistency, and transparency of automated grading, aligning more closely with human grading standards. The results suggest that this framework could be a valuable tool for automated programming assessment systems, offering clear, actionable feedback and making machine learning models in educational contexts more interpretable and effective.<\/jats:p>","DOI":"10.3390\/make7020041","type":"journal-article","created":{"date-parts":[[2025,4,29]],"date-time":"2025-04-29T05:17:12Z","timestamp":1745903832000},"page":"41","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Automated Grading Through Contrastive Learning: A Gradient Analysis and Feature Ablation Approach"],"prefix":"10.3390","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9896-1544","authenticated-orcid":false,"given":"Mateo","family":"Soka\u010d","sequence":"first","affiliation":[{"name":"Department of Software Engineering, Algebra University, Gradi\u0161\u0107anska 24, 10000 Zagreb, Croatia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mario","family":"Fabijani\u0107","sequence":"additional","affiliation":[{"name":"Department of Software Engineering, Algebra University, Gradi\u0161\u0107anska 24, 10000 Zagreb, Croatia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9813-1935","authenticated-orcid":false,"given":"Igor","family":"Mekterovi\u0107","sequence":"additional","affiliation":[{"name":"Faculty of Electrical Engineering and Computing, University of Zagreb, 10000 Zagreb, Croatia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5093-3453","authenticated-orcid":false,"given":"Leo","family":"Mr\u0161i\u0107","sequence":"additional","affiliation":[{"name":"Department of Software Engineering, Algebra University, Gradi\u0161\u0107anska 24, 10000 Zagreb, Croatia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,4,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Tharmaseelan, J., Manathunga, K., Reyal, S., Kasthurirathna, D., and Thurairasa, T. (2021, January 21\u201323). Revisit of automated marking techniques for programming assignments. Proceedings of the 2021 IEEE Global Engineering Education Conference (EDUCON), Vienna, Austria.","DOI":"10.1109\/EDUCON46332.2021.9453889"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"142","DOI":"10.1145\/1473195.1473239","article-title":"Enhancing learning management systems to better support computer science education","volume":"40","author":"Joy","year":"2008","journal-title":"ACM SIGCSE Bull."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1109\/TE.2018.2805706","article-title":"A closer look at the differences between graders in introductory computer science exams","volume":"61","author":"Albluwi","year":"2018","journal-title":"IEEE Trans. Educ."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"81154","DOI":"10.1109\/ACCESS.2020.2990980","article-title":"Building a comprehensive automated programming assessment system","volume":"8","author":"Mekterovic","year":"2020","journal-title":"IEEE Access"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"528","DOI":"10.1145\/367415.367422","article-title":"Automatic graders for programming classes","volume":"3","author":"Hollingsworth","year":"1960","journal-title":"Commun. ACM"},{"key":"ref_6","first-page":"59","article-title":"Automatic assessment of Java code","volume":"53","author":"Insa","year":"2018","journal-title":"Comput. Lang. Syst. Struct."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Sooksatra, K., Khanal, B., Rivas, P., and Schwartz, D.R. (2023, January 13\u201315). Attribution scores of BERT-based SQL-query automatic grading for explainability. Proceedings of the 2023 International Conference on Computational Science and Computational Intelligence (CSCI), Las Vegas, NV, USA.","DOI":"10.1109\/CSCI62032.2023.00039"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Rivas, P., and Schwartz, D.R. (2021, January 15\u201317). Modeling SQL Statement Correctness with Attention-Based Convolutional Neural Networks. Proceedings of the 2021 International Conference on Computational Science and Computational Intelligence (CSCI), Las Vegas, NV, USA.","DOI":"10.1109\/CSCI54926.2021.00086"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3513140","article-title":"Automated assessment in computer science education: A state-of-the-art review","volume":"22","author":"Paiva","year":"2022","journal-title":"ACM Trans. Comput. Educ."},{"key":"ref_10","unstructured":"Kelkar, A., Relan, R., Bhardwaj, V., Vaichal, S., Khatri, C., and Relan, P. (2020). Bertrand-DR: Improving Text-to-SQL using a Discriminative Re-ranker. arXiv."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"3602","DOI":"10.14778\/3681954.3682024","article-title":"QED: A powerful query equivalence decider for SQL","volume":"17","author":"Wang","year":"2024","journal-title":"Proc. VLDB Endow."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1071","DOI":"10.1145\/3649849","article-title":"VeriEQL: Bounded equivalence verification for complex SQL queries with integrity constraints","volume":"8","author":"He","year":"2024","journal-title":"Proc. ACM Program Lang."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Stajduhar, I., and Mausa, G. (2015, January 25\u201329). Using string similarity metrics for automated grading of SQL statements. Proceedings of the 2015 38th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), Opatija, Croatia.","DOI":"10.1109\/MIPRO.2015.7160467"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Bhangdiya, A., Chandra, B., Kar, B., Radhakrishnan, B., Reddy, K.V.M., Shah, S., and Sudarshan, S. (2015, January 13\u201317). The XDa-TA system for automated grading of SQL query assignments. Proceedings of the 2015 IEEE 31st International Conference on Data Engineering, Seoul, Republic of Korea.","DOI":"10.1109\/ICDE.2015.7113403"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Chandra, B., Banerjee, A., Hazra, U., Joseph, M., and Sudarshan, S. (2019, January 8\u201311). Automated grading of SQL queries. Proceedings of the 2019 IEEE 35th International Conference on Data Engineering (ICDE), Macao.","DOI":"10.1109\/ICDE.2019.00159"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1541","DOI":"10.14778\/3007263.3007304","article-title":"Partial marking for automated grading of SQL queries","volume":"9","author":"Chandra","year":"2016","journal-title":"Proc. VLDB Endow."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"731","DOI":"10.1007\/s00778-015-0395-0","article-title":"Data generation for testing and grading SQL queries","volume":"24","author":"Chandra","year":"2015","journal-title":"VLDB J."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Chandra, B., Chawda, B., Shah, S., Sudarshan, S., and Shah, A. (2013, January 24). Extending XData to kill SQL query mutants in the wild. Proceedings of the Sixth International Workshop on Testing Database Systems, New York, NY, USA.","DOI":"10.1145\/2479440.2479442"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"3713","DOI":"10.1007\/s11042-022-13428-4","article-title":"Natural language processing: State of the art, current trends and challenges","volume":"82","author":"Khurana","year":"2023","journal-title":"Multimed. Tools Appl."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Sung, C., Dhamecha, T., Saha, S., Ma, T., Reddy, V., and Arora, R. (2019, January 3\u20137). Pre-training BERT on domain resources for short answer grading. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China.","DOI":"10.18653\/v1\/D19-1628"},{"key":"ref_21","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2018). BERT: Pre-training of deep bidirectional Transformers for language understanding. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1186\/s40537-022-00564-9","article-title":"The performance of BERT as data representation of text clustering","volume":"9","author":"Subakti","year":"2022","journal-title":"J. Big Data"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"M\u00fcller, M., Salath\u00e9, M., and Kummervold, P.E. (2023). COVID-Twitter-BERT: A natural language processing model to analyse COVID-19 content on Twitter. Front. Artif. Intell., 6.","DOI":"10.3389\/frai.2023.1023281"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Chalkidis, I., Fergadiotis, M., Malakasiotis, P., Aletras, N., and Androutsopoulos, I. (2020, January 16\u201320). LEGAL-BERT: The Muppets straight out of Law School. Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event.","DOI":"10.18653\/v1\/2020.findings-emnlp.261"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Abdul Salam, M., El-Fatah, M.A., and Hassan, N.F. (2022). Automatic grading for Arabic short answer questions using optimized deep learning model. PLoS ONE, 17.","DOI":"10.1371\/journal.pone.0272269"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Rahaman, M.A., and Mahmud, H. (2022). Automated evaluation of handwritten answer script using deep learning approach. Trans. Mach. Learn. Artif. Intell., 10.","DOI":"10.14738\/tmlai.104.12831"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Maji, S., Appe, A., Bali, R., Chowdhury, A.G., Raghavendra, V.C., and Bhandaru, V.M. (2021, January 1\u20133). An Interpretable Deep Learning System for Automatically Scoring Request for Proposals. Proceedings of the 2021 IEEE 33rd International Conference on Tools with Artificial Intelligence (ICTAI), Washington, DC, USA.","DOI":"10.1109\/ICTAI52525.2021.00136"},{"key":"ref_28","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"317","DOI":"10.1016\/j.ins.2019.03.080","article-title":"Self-Attention Convolutional Neural Network for Improved MR Image Reconstruction","volume":"490","author":"Wu","year":"2019","journal-title":"Inf. Sci."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"106437","DOI":"10.1016\/j.epsr.2020.106437","article-title":"Self attention convolutional neural network with time series imaging based feature extraction for transmission line fault detection and classification","volume":"187","author":"Fahim","year":"2020","journal-title":"Electr. Power Syst. Res."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1016\/j.specom.2018.09.002","article-title":"Towards automatic assessment of spontaneous spoken English","volume":"104","author":"Wang","year":"2018","journal-title":"Speech Commun."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Rivas, P., Schwartz, D.R., and Quevedo, E. (2023, January 24\u201327). BERT goes to SQL school: Improving automatic grading of SQL statements. Proceedings of the 2023 Congress in Computer Science, Computer Engineering, & Applied Computing (CSCE), Las Vegas, NV, USA.","DOI":"10.1109\/CSCE60160.2023.00019"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Messer, M., Brown, N.C., K\u00f6lling, M., and Shi, M. (2024). How consistent are humans when grading programming assignments?. arXiv.","DOI":"10.35542\/osf.io\/nd6qy"},{"key":"ref_34","unstructured":"Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. (2020, January 13\u201318). A simple framework for contrastive learning of visual representations. Proceedings of the 37th International Conference on Machine Learning, Virtual Event."},{"key":"ref_35","unstructured":"Sundararajan, M., Taly, A., and Yan, Q. (2017, January 6\u201311). Axiomatic attribution for deep networks. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Hong, I., Tran, H., and Donnat, C. (November, January 29). A Simplified Framework for Contrastive Learning for Node Representations. Proceedings of the 2023 57th Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, USA.","DOI":"10.1109\/IEEECONF59524.2023.10476838"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"861","DOI":"10.21105\/joss.00861","article-title":"UMAP: Uniform Manifold Approximation and Projection","volume":"3","author":"McInnes","year":"2018","journal-title":"J. Open Source Softw."},{"key":"ref_38","first-page":"179","article-title":"Combining dynamic and static analysis for automated grading SQL statements","volume":"5","author":"Wang","year":"2020","journal-title":"J. Netw. Intell."},{"key":"ref_39","unstructured":"Shrikumar, A., Greenside, P., and Kundaje, A. (2017, January 6\u201311). Learning important features through propagating activation differences. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia."}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/7\/2\/41\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:23:50Z","timestamp":1760030630000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/7\/2\/41"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,29]]},"references-count":39,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["make7020041"],"URL":"https:\/\/doi.org\/10.3390\/make7020041","relation":{},"ISSN":["2504-4990"],"issn-type":[{"value":"2504-4990","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,29]]}}}