{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,15]],"date-time":"2025-11-15T14:01:41Z","timestamp":1763215301496,"version":"3.45.0"},"reference-count":57,"publisher":"Association for Computing Machinery (ACM)","issue":"1","funder":[{"DOI":"10.13039\/501100002911","name":"Complutense University of Madrid","doi-asserted-by":"crossref","award":["INNOVA-Docencia 2020-21\/18 and 2021-22\/387"],"award-info":[{"award-number":["INNOVA-Docencia 2020-21\/18 and 2021-22\/387"]}],"id":[{"id":"10.13039\/501100002911","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Educ."],"published-print":{"date-parts":[[2026,3,31]]},"abstract":"<jats:p>Databases are a key topic in many technical university degrees. As databases have a strong practical nature, students are expected to solve many exercises before mastering the different aspects involved: querying and modifying the database, writing procedural code (functions and procedures), and defining triggers, among others. In this scenario, it is very important to have a substantial number of exercises available but also a timely feedback to detect and fix mistakes. Therefore, automatic judges that execute students\u2019 solutions and generate immediate feedback are valuable tools to include in the teaching practice.<\/jats:p>\n                  <jats:p>In this article, we assess the real impact of using an online automatic judge for free practice in a database course over four academic years. For this purpose, we have contrasted the marks obtained in one academic year, without the automatic judge, against the three following years in which the automatic judge was used. The results show that final marks are statistically higher during the years when students make use of the automatic judge, thus showing an overall positive impact on database learning. Similarly, the results show that the more students use the automatic judge, the higher their final marks are. Besides these two insights, we have also studied if the impact of the automatic judge is the same in groups of high-profile students, concluding that this tool is less effective when improving learning in top-performing, highly self-motivated students in a database course.<\/jats:p>","DOI":"10.1145\/3769852","type":"journal-article","created":{"date-parts":[[2025,9,30]],"date-time":"2025-09-30T15:25:37Z","timestamp":1759245937000},"page":"1-37","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["LearnSQL: Impact of an Automatic Judge in Database Learning"],"prefix":"10.1145","volume":"26","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1664-018X","authenticated-orcid":false,"given":"Enrique","family":"Martin-Martin","sequence":"first","affiliation":[{"name":"Dpto. Sistemas Inform\u00e1ticos y Computaci\u00f3n, Facultad de Inform\u00e1tica, Universidad Complutense de Madrid, Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9686-465X","authenticated-orcid":false,"given":"Manuel","family":"Montenegro","sequence":"additional","affiliation":[{"name":"Dpto. Sistemas Inform\u00e1ticos y Computaci\u00f3n, Facultad de Inform\u00e1tica, Universidad Complutense de Madrid, Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9716-4612","authenticated-orcid":false,"given":"Adri\u00e1n","family":"Riesco","sequence":"additional","affiliation":[{"name":"Dpto. Sistemas Inform\u00e1ticos y Computaci\u00f3n, Facultad de Inform\u00e1tica, Universidad Complutense de Madrid, Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2983-3404","authenticated-orcid":false,"given":"Rub\u00e9n","family":"Rubio","sequence":"additional","affiliation":[{"name":"Dpto. Sistemas Inform\u00e1ticos y Computaci\u00f3n, Facultad de Inform\u00e1tica, Universidad Complutense de Madrid, Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6075-4398","authenticated-orcid":false,"given":"Fernando","family":"S\u00e1enz-P\u00e9rez","sequence":"additional","affiliation":[{"name":"Dpto. Ingenier\u00eda del Software e Inteligencia Artificial, Facultad de Inform\u00e1tica, Universidad Complutense de Madrid, Madrid, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,11,15]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/2839509.2844640"},{"key":"e_1_3_2_3_2","volume-title":"A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom\u2019s Taxonomy of Educational Objectives: Complete Edition","author":"Anderson Lorin W.","year":"2001","unstructured":"Lorin W. Anderson and David R. Krathwohl. 2001. A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom\u2019s Taxonomy of Educational Objectives: Complete Edition. Addison Wesley Longman, Inc."},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","unstructured":"Enrique Martin-Martin Manuel Montenegro Adri\u00e1n Riesco Rub\u00e9n Rubio and Fernando S\u00e1enz-P\u00e9rez. 2024. LearnSQL: Impact of an Automatic Judge in Database Learning (Dataset). DOI: 10.5281\/zenodo.11181272","DOI":"10.5281\/zenodo.11181272"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0234685"},{"key":"e_1_3_2_6_2","volume-title":"Taxonomy of Educational Objectives: The Classification of Educational Goals. Handbook I: Cognitive Domain","author":"Bloom Benjamin S.","year":"1956","unstructured":"Benjamin S. Bloom, Max D. Engelhart, Edward J. Furst, Walker H. Hill, and David R. Krathwohl. 1956. Taxonomy of Educational Objectives: The Classification of Educational Goals. Handbook I: Cognitive Domain. David McKay Company."},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2005.06.028"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1080\/10494820.2018.1451345"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/COMPSAC.2018.00174"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/COMPSAC61105.2024.00021"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/TALE56641.2023.10398239"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.4324\/9780203771587"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jaac.2020.08.466"},{"key":"e_1_3_2_14_2","unstructured":"Andrew Cumming. 2020. SQL Zoo. Intelligent Growth Solutions. Retrieved from https:\/\/sqlzoo.net\/"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.18848\/1447-9494\/CGP\/v12i03\/46714"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/1315803.1315821"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3264507"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2010.05.026"},{"key":"e_1_3_2_19_2","volume-title":"Fundamentals of Database Systems","author":"Elmasri Ramez","year":"2017","unstructured":"Ramez Elmasri and Shamkant B. Navathe. 2017. Fundamentals of Database Systems (7th. ed.). Pearson Harlow.","edition":"7"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3467967"},{"key":"e_1_3_2_21_2","volume-title":"DOMjudge 8.2 Manual","author":"Gerritsen Nicky","year":"2025","unstructured":"Nicky Gerritsen, Thijs Kinkhorst, and Tobias Werth. 2025. DOMjudge 8.2 Manual. DOMjudge."},{"issue":"1","key":"e_1_3_2_22_2","first-page":"53","article-title":"An effective approach for teaching database course","volume":"12","author":"Sastry Musti K. S.","year":"2015","unstructured":"Musti K. S. Sastry. 2015. An effective approach for teaching database course. International Journal of Learning, Teaching and Educational Research 12, 1 (2015), 53\u201363.","journal-title":"International Journal of Learning, Teaching and Educational Research"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3197091.3197124"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/2526968.2526986"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.14742\/AJET.1036"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2021.669119"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.18293\/SEKE2022-025"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3446871.3469759"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524610.3529158"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.11120\/ital.2010.09010032"},{"key":"e_1_3_2_31_2","first-page":"238","article-title":"Evaluation of a constraint-based tutor for a database language","volume":"10","author":"Mitrovic Antonija","year":"1999","unstructured":"Antonija Mitrovic and Stellan Ohlsson. 1999. Evaluation of a constraint-based tutor for a database language. International Journal of Artificial Intelligence in Education 10 (1999), 238\u2013256.","journal-title":"International Journal of Artificial Intelligence in Education"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1007\/S40593-015-0049-9"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3427597"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-662-03037-0_7"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/1007996.1008007"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3513140"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1111\/bjet.12953"},{"key":"e_1_3_2_38_2","first-page":"26","volume-title":"2015 ASEE Annual Conference & Exposition","author":"Pettit Raymond Scott","year":"2015","unstructured":"Raymond Scott Pettit, John D. Homer, Kayla Michelle McMurry, Nevan Simone, and Susan A. Mengel. 2015. Are automated assessment tools helpful in programming courses? In 2015 ASEE Annual Conference & Exposition, 26\u2013230."},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIET60671.2024.10542806"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3341525.3387391"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3663649.3664371"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/2591708.2602682"},{"key":"e_1_3_2_43_2","first-page":"281","volume-title":"Cognition and Exploratory Learning in Digital Age (CELDA \u201904), IADIS International Conference","author":"Russell Gordon","year":"2004","unstructured":"Gordon Russell and Andrew Cumming. 2004. Improving the student learning experience for SQL using automatic marking. In Cognition and Exploratory Learning in Digital Age (CELDA \u201904), IADIS International Conference. Kinshuk, Demetrios G. Sampson, and Pedro T. Isa\u00edas (Eds.), IADIS, 281\u2013288."},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/1007996.1008055"},{"key":"e_1_3_2_45_2","first-page":"539","volume-title":"Handbook of Complementary Methods in Education Research","author":"Shadish William R.","year":"2012","unstructured":"William R. Shadish and Jason K. Luellen. 2012. Quasi-experimental design. In Handbook of Complementary Methods in Education Research. Routledge, 539\u2013550."},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10956-013-9437-9"},{"key":"e_1_3_2_47_2","volume-title":"Database System Concepts","author":"Silberschatz Abraham","year":"2019","unstructured":"Abraham Silberschatz, Henry F. Korth, and Shashank Sudarshan. 2019. Database System Concepts (7th. ed.). McGraw-Hill.","edition":"7"},{"key":"e_1_3_2_48_2","volume-title":"International Conference on Information Technology Based Higher Education and Training, ITHET","author":"Soler Josep","year":"2006","unstructured":"Josep Soler, Ferran Prados, Imma Boada, and Jordi Poch. 2006. A web-based tool for teaching and learning SQL. In International Conference on Information Technology Based Higher Education and Training, ITHET."},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1177\/0956797617741719"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.entcs.2011.02.011"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1017\/S1471068419000206"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1186\/s41039-022-00180-4"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/3596673.3603142"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2021.111034"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/3398377"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1145\/3231712"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3654677"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.2196\/22817"}],"container-title":["ACM Transactions on Computing Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3769852","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,15]],"date-time":"2025-11-15T13:57:07Z","timestamp":1763215027000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3769852"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,15]]},"references-count":57,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,3,31]]}},"alternative-id":["10.1145\/3769852"],"URL":"https:\/\/doi.org\/10.1145\/3769852","relation":{},"ISSN":["1946-6226"],"issn-type":[{"type":"electronic","value":"1946-6226"}],"subject":[],"published":{"date-parts":[[2025,11,15]]},"assertion":[{"value":"2024-05-24","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-09-18","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-11-15","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}