{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,30]],"date-time":"2026-01-30T05:40:32Z","timestamp":1769751632251,"version":"3.49.0"},"publisher-location":"New York, NY, USA","reference-count":32,"publisher":"ACM","license":[{"start":{"date-parts":[[2023,7,12]],"date-time":"2023-07-12T00:00:00Z","timestamp":1689120000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,7,12]]},"DOI":"10.1145\/3597926.3598049","type":"proceedings-article","created":{"date-parts":[[2023,7,13]],"date-time":"2023-07-13T20:12:53Z","timestamp":1689279173000},"page":"199-210","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Concept-Based Automated Grading of CS-1 Programming Assignments"],"prefix":"10.1145","author":[{"given":"Zhiyu","family":"Fan","sequence":"first","affiliation":[{"name":"National University of Singapore, Singapore"}]},{"given":"Shin Hwei","family":"Tan","sequence":"additional","affiliation":[{"name":"Concordia University, Canada"}]},{"given":"Abhik","family":"Roychoudhury","sequence":"additional","affiliation":[{"name":"National University of Singapore, Singapore"}]}],"member":"320","published-online":{"date-parts":[[2023,7,13]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Verifix: Verified repair of programming assignments. ACM Transactions on Software Engineering and Methodology.","author":"Ahmed Umair Z","year":"2022","unstructured":"Umair Z Ahmed , Zhiyu Fan , Jooyong Yi , Omar I Al-Bataineh , and Abhik Roychoudhury . 2022 . Verifix: Verified repair of programming assignments. ACM Transactions on Software Engineering and Methodology. Umair Z Ahmed, Zhiyu Fan, Jooyong Yi, Omar I Al-Bataineh, and Abhik Roychoudhury. 2022. Verifix: Verified repair of programming assignments. ACM Transactions on Software Engineering and Methodology."},{"key":"e_1_3_2_1_2_1","volume-title":"Proceedings of the 6th Baltic Sea conference on Computing education research: Koli Calling","author":"Ahoniemi Tuukka","year":"2006","unstructured":"Tuukka Ahoniemi and Tommi Reinikainen . 2006 . ALOHA-a grading tool for semi-automatic assessment of mass programming courses . In Proceedings of the 6th Baltic Sea conference on Computing education research: Koli Calling 2006. 139\u2013140. Tuukka Ahoniemi and Tommi Reinikainen. 2006. ALOHA-a grading tool for semi-automatic assessment of mass programming courses. In Proceedings of the 6th Baltic Sea conference on Computing education research: Koli Calling 2006. 139\u2013140."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.28945\/300"},{"key":"e_1_3_2_1_4_1","volume-title":"Proceedings of the 51st ACM technical symposium on computer science education. 556\u2013562","author":"da Cruz Alves Nathalia","year":"2020","unstructured":"Nathalia da Cruz Alves , Christiane Gresse von Wangenheim , Jean Carlo Rossa Hauck , and Adriano Ferreti Borgatto . 2020 . A large-scale evaluation of a rubric for the automatic assessment of algorithms and programming concepts . In Proceedings of the 51st ACM technical symposium on computer science education. 556\u2013562 . Nathalia da Cruz Alves, Christiane Gresse von Wangenheim, Jean Carlo Rossa Hauck, and Adriano Ferreti Borgatto. 2020. A large-scale evaluation of a rubric for the automatic assessment of algorithms and programming concepts. In Proceedings of the 51st ACM technical symposium on computer science education. 556\u2013562."},{"key":"e_1_3_2_1_5_1","volume-title":"Formative assessment and science education. 12","author":"Bell Beverley","unstructured":"Beverley Bell , Nigel Bell , and B Cowie . 2001. Formative assessment and science education. 12 , Springer Science & Business Media . Beverley Bell, Nigel Bell, and B Cowie. 2001. Formative assessment and science education. 12, Springer Science & Business Media."},{"key":"e_1_3_2_1_6_1","volume-title":"Joint IAPR International Workshops on Statistical Techniques in Pattern Recognition (SPR) and Structural and Syntactic Pattern Recognition (SSPR). 123\u2013132","author":"Bunke Horst","year":"2002","unstructured":"Horst Bunke , Pasquale Foggia , Corrado Guidobaldi , Carlo Sansone , and Mario Vento . 2002 . A comparison of algorithms for maximum common subgraph on randomly connected graphs . In Joint IAPR International Workshops on Statistical Techniques in Pattern Recognition (SPR) and Structural and Syntactic Pattern Recognition (SSPR). 123\u2013132 . Horst Bunke, Pasquale Foggia, Corrado Guidobaldi, Carlo Sansone, and Mario Vento. 2002. A comparison of algorithms for maximum common subgraph on randomly connected graphs. In Joint IAPR International Workshops on Statistical Techniques in Pattern Recognition (SPR) and Structural and Syntactic Pattern Recognition (SSPR). 123\u2013132."},{"key":"e_1_3_2_1_7_1","unstructured":"coetaur0. 2019. Python3 control flow graph generator. https:\/\/github.com\/coetaur0\/staticfg Accessed: 2022-10-09 \t\t\t\t  coetaur0. 2019. Python3 control flow graph generator. https:\/\/github.com\/coetaur0\/staticfg Accessed: 2022-10-09"},{"key":"e_1_3_2_1_8_1","unstructured":"Berkeley EECS. [n. d.]. CS 9H: Python for Programmers. https:\/\/selfpaced.bitbucket.io\/#\/python\/calendar \t\t\t\t  Berkeley EECS. [n. d.]. CS 9H: Python for Programmers. https:\/\/selfpaced.bitbucket.io\/#\/python\/calendar"},{"key":"e_1_3_2_1_9_1","unstructured":"Stanford Engineering. [n. d.]. CS106A - Programming Methodology. https:\/\/web.stanford.edu\/class\/archive\/cs\/cs106a\/cs106a.1206\/schedule.html \t\t\t\t  Stanford Engineering. [n. d.]. CS106A - Programming Methodology. https:\/\/web.stanford.edu\/class\/archive\/cs\/cs106a\/cs106a.1206\/schedule.html"},{"key":"e_1_3_2_1_10_1","unstructured":"Computer Science for ALL Students. 2022. AutoGradr. https:\/\/www.csforall.org\/members\/autogradr_automated_grading_for_programming_assignments\/ Accessed: 2020-10-06 \t\t\t\t  Computer Science for ALL Students. 2022. AutoGradr. https:\/\/www.csforall.org\/members\/autogradr_automated_grading_for_programming_assignments\/ Accessed: 2020-10-06"},{"key":"e_1_3_2_1_11_1","unstructured":"Python Software Foundation. 2022. Abstract Syntax Trees. https:\/\/docs.python.org\/3\/library\/ast.html \t\t\t\t  Python Software Foundation. 2022. Abstract Syntax Trees. https:\/\/docs.python.org\/3\/library\/ast.html"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2017.2664836"},{"key":"e_1_3_2_1_13_1","volume-title":"Auto-grading dynamic programming language assignments","author":"Gong Liang","unstructured":"Liang Gong . 2014. Auto-grading dynamic programming language assignments . University of California , Berkeley, Tech . Rep. Liang Gong. 2014. Auto-grading dynamic programming language assignments. University of California, Berkeley, Tech. Rep."},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3296979.3192387"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/362946.362981"},{"key":"e_1_3_2_1_16_1","volume-title":"2019 34th IEEE\/ACM International Conference on Automated Software Engineering (ASE). 388\u2013398","author":"Hu Yang","year":"2019","unstructured":"Yang Hu , Umair Z. Ahmed , Sergey Mechtaev , Ben Leong , and Abhik Roychoudhury . 2019 . Re-factoring based Program Repair applied to Programming Assignments . In 2019 34th IEEE\/ACM International Conference on Automated Software Engineering (ASE). 388\u2013398 . Yang Hu, Umair Z. Ahmed, Sergey Mechtaev, Ben Leong, and Abhik Roychoudhury. 2019. Re-factoring based Program Repair applied to Programming Assignments. In 2019 34th IEEE\/ACM International Conference on Automated Software Engineering (ASE). 388\u2013398."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1930464.1930480"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1163405.1163407"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2950290.2950363"},{"key":"e_1_3_2_1_20_1","volume-title":"2019 IEEE\/ACM 41st International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET). 126\u2013137","author":"Liu Xiao","year":"2019","unstructured":"Xiao Liu , Shuai Wang , Pei Wang , and Dinghao Wu . 2019 . Automatic grading of programming assignments: an approach based on formal semantics . In 2019 IEEE\/ACM 41st International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET). 126\u2013137 . Xiao Liu, Shuai Wang, Pei Wang, and Dinghao Wu. 2019. Automatic grading of programming assignments: an approach based on formal semantics. In 2019 IEEE\/ACM 41st International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET). 126\u2013137."},{"key":"e_1_3_2_1_21_1","volume-title":"Software Engineering Conference, Australian. 38\u201338","author":"Michaelson Greg","year":"1996","unstructured":"Greg Michaelson . 1996 . Automatic analysis of functional program style . In Software Engineering Conference, Australian. 38\u201338 . Greg Michaelson. 1996. Automatic analysis of functional program style. In Software Engineering Conference, Australian. 38\u201338."},{"key":"e_1_3_2_1_22_1","unstructured":"MIT OpenCourseWare. [n. d.]. 6.0001 Introduction to Computer Science and Programming in Python. https:\/\/ocw.mit.edu\/courses\/ \t\t\t\t  MIT OpenCourseWare. [n. d.]. 6.0001 Introduction to Computer Science and Programming in Python. https:\/\/ocw.mit.edu\/courses\/"},{"key":"e_1_3_2_1_23_1","volume-title":"A socio-cultural theorisation of formative assessment. Oxford review of Education, 34, 1","author":"Pryor John","year":"2008","unstructured":"John Pryor and Barbara Crossouard . 2008. A socio-cultural theorisation of formative assessment. Oxford review of Education, 34, 1 ( 2008 ), 1\u201320. John Pryor and Barbara Crossouard. 2008. A socio-cultural theorisation of formative assessment. Oxford review of Education, 34, 1 (2008), 1\u201320."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2491956.2462195"},{"key":"e_1_3_2_1_25_1","unstructured":"Carnegie Mellon University. [n. d.]. CMU 15-122 Fundamentals of Programming and Computer Science. https:\/\/www.cs.cmu.edu\/ 112\/schedule.html \t\t\t\t  Carnegie Mellon University. [n. d.]. CMU 15-122 Fundamentals of Programming and Computer Science. https:\/\/www.cs.cmu.edu\/ 112\/schedule.html"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1145\/182107.182101","article-title":"Kassandra: the automatic grading system","volume":"22","author":"Matt Urs Von","year":"1994","unstructured":"Urs Von Matt . 1994 . Kassandra: the automatic grading system . ACM SIGCUE Outlook , 22 , 1 (1994), 26 \u2013 40 . Urs Von Matt. 1994. Kassandra: the automatic grading system. ACM SIGCUE Outlook, 22, 1 (1994), 26\u201340.","journal-title":"ACM SIGCUE Outlook"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2012.12.005"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3192366.3192384"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/1047124.1047427"},{"key":"e_1_3_2_1_30_1","volume-title":"The Free Encyclopedia. https:\/\/en.wikipedia.org\/w\/index.php?title=Root-mean-square_deviation&oldid=1117272661 [Online","author":"Wikipedia","year":"2022","unstructured":"Wikipedia contributors. 2022. Root-mean-square deviation \u2014 Wikipedia , The Free Encyclopedia. https:\/\/en.wikipedia.org\/w\/index.php?title=Root-mean-square_deviation&oldid=1117272661 [Online ; accessed 10- November - 2022 ] Wikipedia contributors. 2022. Root-mean-square deviation \u2014 Wikipedia, The Free Encyclopedia. https:\/\/en.wikipedia.org\/w\/index.php?title=Root-mean-square_deviation&oldid=1117272661 [Online; accessed 10-November-2022]"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3106237.3106262"},{"key":"e_1_3_2_1_32_1","volume-title":"Proceedings of the 2018 conference on empirical methods in natural language processing.","author":"Zhong Zexuan","year":"2018","unstructured":"Zexuan Zhong , Jiaqi Guo , Wei Yang , Jian Peng , Tao Xie , Jian-Guang Lou , Ting Liu , and Dongmei Zhang . 2018 . SemRegex: A semantics-based approach for generating regular expressions from natural language specifications . In Proceedings of the 2018 conference on empirical methods in natural language processing. Zexuan Zhong, Jiaqi Guo, Wei Yang, Jian Peng, Tao Xie, Jian-Guang Lou, Ting Liu, and Dongmei Zhang. 2018. SemRegex: A semantics-based approach for generating regular expressions from natural language specifications. In Proceedings of the 2018 conference on empirical methods in natural language processing."}],"event":{"name":"ISSTA '23: 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis","location":"Seattle WA USA","acronym":"ISSTA '23","sponsor":["SIGSOFT ACM Special Interest Group on Software Engineering","AITO"]},"container-title":["Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3597926.3598049","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3597926.3598049","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:48:41Z","timestamp":1750182521000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3597926.3598049"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,12]]},"references-count":32,"alternative-id":["10.1145\/3597926.3598049","10.1145\/3597926"],"URL":"https:\/\/doi.org\/10.1145\/3597926.3598049","relation":{},"subject":[],"published":{"date-parts":[[2023,7,12]]},"assertion":[{"value":"2023-07-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}