{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,27]],"date-time":"2026-05-27T13:00:34Z","timestamp":1779886834612,"version":"3.53.1"},"publisher-location":"New York, NY, USA","reference-count":35,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,11,22]],"date-time":"2021-11-22T00:00:00Z","timestamp":1637539200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,11,22]]},"DOI":"10.1145\/3507923.3507956","type":"proceedings-article","created":{"date-parts":[[2022,4,13]],"date-time":"2022-04-13T23:32:05Z","timestamp":1649892725000},"page":"94-107","source":"Crossref","is-referenced-by-count":5,"title":["What does this Python code do? An exploratory analysis of novice students\u2019 code explanations"],"prefix":"10.1145","author":[{"given":"Vivian","family":"van der Werf","sequence":"first","affiliation":[{"name":"Leiden University, Leiden Institute of Advanced Computer Science, Netherlands and Delft University of Technology, Leiden-Delft-Erasmus Centre for Education and Learning, Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Efthimia","family":"Aivaloglou","sequence":"additional","affiliation":[{"name":"Leiden University, Leiden Institute of Advanced Computer Science, Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Felienne","family":"Hermans","sequence":"additional","affiliation":[{"name":"Leiden University, Leiden Institute of Advanced Computer Science, Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marcus","family":"Specht","sequence":"additional","affiliation":[{"name":"Delft University of Technology, Leiden-Delft-Erasmus Centre for Education and Learning, Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,4,13]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Evaluating the Quality of Learning. The SOLO Taxonomy","author":"Biggs B.","year":"2013","unstructured":"John.\u00a0 B. Biggs and Kevin\u00a0 F. Collins . 1982. Evaluating the Quality of Learning. The SOLO Taxonomy . Elsevier Inc .https:\/\/doi.org\/10.1016\/c 2013 -0-10375-3 Publication Title : Educational Psychology . John.\u00a0B. Biggs and Kevin\u00a0F. Collins. 1982. Evaluating the Quality of Learning. The SOLO Taxonomy.Elsevier Inc.https:\/\/doi.org\/10.1016\/c2013-0-10375-3 Publication Title: Educational Psychology."},{"key":"e_1_3_2_1_2_1","volume-title":"New frameworks for studying and assessing the development of computational thinking","author":"Brennan Karen","year":"1860","unstructured":"Karen Brennan and Mitchel Resnick . 2012. New frameworks for studying and assessing the development of computational thinking . In American Educational Research Association . https:\/\/doi.org\/10.1007\/978-3-319-64051-8_9 ISSN: 1860 949X. Karen Brennan and Mitchel Resnick. 2012. New frameworks for studying and assessing the development of computational thinking. In American Educational Research Association. https:\/\/doi.org\/10.1007\/978-3-319-64051-8_9 ISSN: 1860949X."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-7373(83)80031-5"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2526968.2526969"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3328778.3366879"},{"key":"e_1_3_2_1_6_1","volume-title":"Beth Simon, and Errol Thompson.","author":"Clear Tony","year":"2008","unstructured":"Tony Clear , Jacqueline L Whalley , Raymond Lister , Angela Carbone , Minjie Hu , Judithe Irene Sheard , Beth Simon, and Errol Thompson. 2008 . Reliably classifying novice programmer exam responses using the SOLO taxonomy. In Proceedings of the Twenty First Annual Conference of the National Advisory Committee on Computing Qualifications, Samuel Mannand Mike Lopez (Eds.). National Advisory Committee on Computing Qualifications (NACCQ), 23 \u2013 30. 21st National Advisory Committee on Computing Qualifications, NACCQ 2008 ; Conference date: 04-07-2008 Through 07-07-2008. Tony Clear, Jacqueline L Whalley, Raymond Lister, Angela Carbone, Minjie Hu, Judithe Irene Sheard, Beth Simon, and Errol Thompson. 2008. Reliably classifying novice programmer exam responses using the SOLO taxonomy. In Proceedings of the Twenty First Annual Conference of the National Advisory Committee on Computing Qualifications, Samuel Mannand Mike Lopez (Eds.). National Advisory Committee on Computing Qualifications (NACCQ), 23 \u2013 30. 21st National Advisory Committee on Computing Qualifications, NACCQ 2008 ; Conference date: 04-07-2008 Through 07-07-2008."},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2538862.2538911"},{"key":"e_1_3_2_1_8_1","volume-title":"Relational Reasoning. In Proceedings of the Thirteenth Australasian Computing Education Conference -","volume":"114","author":"Corney Malcolm","year":"2011","unstructured":"Malcolm Corney , Raymond Lister , and Donna Teague . 2011 . Early Relational Reasoning and the Novice Programmer: Swapping as the \u201dHello World\u201d of Relational Reasoning. In Proceedings of the Thirteenth Australasian Computing Education Conference - Volume 114 (Perth, Australia) (ACE \u201911). Australian Computer Society, Inc., AUS, 95\u2013104. Malcolm Corney, Raymond Lister, and Donna Teague. 2011. Early Relational Reasoning and the Novice Programmer: Swapping as the \u201dHello World\u201d of Relational Reasoning. In Proceedings of the Thirteenth Australasian Computing Education Conference - Volume 114 (Perth, Australia) (ACE \u201911). Australian Computer Society, Inc., AUS, 95\u2013104."},{"key":"e_1_3_2_1_9_1","volume-title":"Proceedings of the Fourteenth Australasian Computing Education Conference -","volume":"123","author":"Corney Malcolm","year":"2012","unstructured":"Malcolm Corney , Donna Teague , Alireza Ahadi , and Raymond Lister . 2012 . Some Empirical Results for Neo-Piagetian Reasoning in Novice Programmers and the Relationship to Code Explanation Questions . In Proceedings of the Fourteenth Australasian Computing Education Conference - Volume 123 (Melbourne, Australia) (ACE \u201912). Australian Computer Society, Inc., AUS, 77\u201386. Malcolm Corney, Donna Teague, Alireza Ahadi, and Raymond Lister. 2012. Some Empirical Results for Neo-Piagetian Reasoning in Novice Programmers and the Relationship to Code Explanation Questions. In Proceedings of the Fourteenth Australasian Computing Education Conference - Volume 123 (Melbourne, Australia) (ACE \u201912). Australian Computer Society, Inc., AUS, 77\u201386."},{"key":"e_1_3_2_1_10_1","volume-title":"Autograding \u201dExplain in Plain English","author":"Fowler Max","unstructured":"Max Fowler , Binglin Chen , Sushmita Azad , Matthew West , and Craig Zilles . 2021. Autograding \u201dExplain in Plain English \u201d Questions Using NLP. Association for Computing Machinery , New York, NY, USA , 1163\u20131169. https:\/\/doi.org\/10.1145\/3408877.3432539 Max Fowler, Binglin Chen, Sushmita Azad, Matthew West, and Craig Zilles. 2021. Autograding \u201dExplain in Plain English\u201d Questions Using NLP. Association for Computing Machinery, New York, NY, USA, 1163\u20131169. https:\/\/doi.org\/10.1145\/3408877.3432539"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3344429.3372501"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3304221.3319751"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1207\/s15326985ep4102_1"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3430665.3456322"},{"key":"e_1_3_2_1_15_1","volume-title":"Automatically. In 2021 IEEE\/ACM 29th International Conference on Program Comprehension (ICPC). Association for Computing Machinery, 467\u2013475","author":"Lehtinen Teemu","year":"2021","unstructured":"Teemu Lehtinen , Andr\u00e9\u00a0 L. Santos , and Juha Sorva . 2021 . Let\u2019s Ask Students About Their Programs , Automatically. In 2021 IEEE\/ACM 29th International Conference on Program Comprehension (ICPC). Association for Computing Machinery, 467\u2013475 . https:\/\/doi.org\/10.1109\/ICPC52881.2021.00054 Teemu Lehtinen, Andr\u00e9\u00a0L. Santos, and Juha Sorva. 2021. Let\u2019s Ask Students About Their Programs, Automatically. In 2021 IEEE\/ACM 29th International Conference on Program Comprehension (ICPC). Association for Computing Machinery, 467\u2013475. https:\/\/doi.org\/10.1109\/ICPC52881.2021.00054"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2978249.2978251"},{"key":"e_1_3_2_1_17_1","volume-title":"On the Cognitive Development of the Novice Programmer: And the Development of a Computing Education Researcher","author":"Lister Raymond","unstructured":"Raymond Lister . 2020. On the Cognitive Development of the Novice Programmer: And the Development of a Computing Education Researcher . Association for Computing Machinery, New York, NY , USA. https:\/\/doi.org\/10.1145\/3442481.3442498 Raymond Lister. 2020. On the Cognitive Development of the Novice Programmer: And the Development of a Computing Education Researcher. Association for Computing Machinery, New York, NY, USA. https:\/\/doi.org\/10.1145\/3442481.3442498"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1562877.1562930"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1140124.1140157"},{"key":"e_1_3_2_1_20_1","volume-title":"Proceedings of the ACM Workshop on International Computing Education Research","author":"Lopez Mike","year":"2008","unstructured":"Mike Lopez , Jacqueline Whalley , Phil Robbins , and Raymond Lister . 2008 . Relationships between reading, tracing and writing skills in introductory programming. In ICER\u201908 - Proceedings of the ACM Workshop on International Computing Education Research ( New York, New York, USA). ACM Press, 101\u2013111. https:\/\/doi.org\/10.1145\/1404520.1404531 Mike Lopez, Jacqueline Whalley, Phil Robbins, and Raymond Lister. 2008. Relationships between reading, tracing and writing skills in introductory programming. In ICER\u201908 - Proceedings of the ACM Workshop on International Computing Education Research (New York, New York, USA). ACM Press, 101\u2013111. https:\/\/doi.org\/10.1145\/1404520.1404531"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2787622.2787733"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2361276.2361299"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2157136.2157249"},{"key":"e_1_3_2_1_24_1","volume-title":"On the Frequency of Words Used in Answers to Explain in Plain English Questions by Novice Programmers","author":"Pelchen Thomas","unstructured":"Thomas Pelchen and Raymond Lister . 2019. On the Frequency of Words Used in Answers to Explain in Plain English Questions by Novice Programmers . Association for Computing Machinery , New York, NY, USA , 11\u201320. https:\/\/doi.org\/10.1145\/3286960.3286962 Thomas Pelchen and Raymond Lister. 2019. On the Frequency of Words Used in Answers to Explain in Plain English Questions by Novice Programmers. Association for Computing Machinery, New York, NY, USA, 11\u201320. https:\/\/doi.org\/10.1145\/3286960.3286962"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3341525.3387379"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3328778.3366833"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/1404520.1404535"},{"key":"e_1_3_2_1_28_1","volume-title":"Proceedings of the Sixteenth Australasian Computing Education Conference -","volume":"148","author":"Sheard Judy","year":"2014","unstructured":"Judy Sheard , Simon, Julian Dermoudy , Daryl D\u2019Souza , Minjie Hu , and Dale Parsons . 2014 . Benchmarking a set of exam questions for introductory programming . In Proceedings of the Sixteenth Australasian Computing Education Conference - Volume 148 . Australian Computer Society, Inc., 113\u2013121. Judy Sheard, Simon, Julian Dermoudy, Daryl D\u2019Souza, Minjie Hu, and Dale Parsons. 2014. Benchmarking a set of exam questions for introductory programming. In Proceedings of the Sixteenth Australasian Computing Education Conference - Volume 148. Australian Computer Society, Inc., 113\u2013121."},{"key":"e_1_3_2_1_29_1","volume-title":"Cognitive Load Theory. Vol.\u00a055","author":"Sweller John","unstructured":"John Sweller . 2011. Cognitive Load Theory. Vol.\u00a055 . Academic Press . https:\/\/doi.org\/10.1016\/B978-0-12-387691-1.00002-8 ISSN: 00797421 Publication Title : Psychology of Learning and Motivation - Advances in Research and Theory. John Sweller. 2011. Cognitive Load Theory. Vol.\u00a055. Academic Press. https:\/\/doi.org\/10.1016\/B978-0-12-387691-1.00002-8 ISSN: 00797421 Publication Title: Psychology of Learning and Motivation - Advances in Research and Theory."},{"key":"e_1_3_2_1_30_1","volume-title":"Ten steps to complex learning. A systematic approach to four-component instructional design. (seconded.)","author":"van Merri\u00ebnboer J.G.","unstructured":"Jeroen\u00a0 J.G. van Merri\u00ebnboer and Paul\u00a0 A. Kirschner . 2013. Ten steps to complex learning. A systematic approach to four-component instructional design. (seconded.) . Routledge . Jeroen\u00a0J.G. van Merri\u00ebnboer and Paul\u00a0A. Kirschner. 2013. Ten steps to complex learning. A systematic approach to four-component instructional design. (seconded.). Routledge."},{"key":"e_1_3_2_1_31_1","volume-title":"Proceedings of the 2009 ACM Workshop on International Computing Education Research","author":"Venables Anne","year":"2009","unstructured":"Anne Venables , Grace Tan , and Raymond Lister . 2009 . A closer look at tracing, explaining and code writing skills in the novice programmer. In ICER\u201909 - Proceedings of the 2009 ACM Workshop on International Computing Education Research ( New York, New York, USA). ACM Press, 117\u2013128. https:\/\/doi.org\/10.1145\/1584322.1584336 Anne Venables, Grace Tan, and Raymond Lister. 2009. A closer look at tracing, explaining and code writing skills in the novice programmer. In ICER\u201909 - Proceedings of the 2009 ACM Workshop on International Computing Education Research (New York, New York, USA). ACM Press, 117\u2013128. https:\/\/doi.org\/10.1145\/1584322.1584336"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3428029.3428031"},{"key":"e_1_3_2_1_33_1","volume-title":"Proceedings of the Sixteenth Australasian Computing Education Conference -","volume":"148","author":"Whalley Jacqueline","year":"2014","unstructured":"Jacqueline Whalley and Nadia Kasto . 2014 . How difficult are novice code writing tasks? a software metrics approach . In Proceedings of the Sixteenth Australasian Computing Education Conference - Volume 148 . Australian Computer Society, Inc., 105\u2013112. Jacqueline Whalley and Nadia Kasto. 2014. How difficult are novice code writing tasks? a software metrics approach. In Proceedings of the Sixteenth Australasian Computing Education Conference - Volume 148. Australian Computer Society, Inc., 105\u2013112."},{"key":"e_1_3_2_1_34_1","volume-title":"Proceedings of the 8th Australasian Conference on Computing Education -","volume":"52","author":"Whalley L.","year":"2006","unstructured":"Jacqueline\u00a0 L. Whalley , Raymond Lister , Errol Thompson , Tony Clear , Phil Robbins , P.\u00a0K.\u00a0 Ajith Kumar , and Christine Prasad . 2006 . An Australasian Study of Reading and Comprehension Skills in Novice Programmers, Using the Bloom and SOLO Taxonomies . In Proceedings of the 8th Australasian Conference on Computing Education - Volume 52 (Hobart, Australia) (ACE \u201906). Australian Computer Society, Inc., AUS, 243\u2013252. Jacqueline\u00a0L. Whalley, Raymond Lister, Errol Thompson, Tony Clear, Phil Robbins, P.\u00a0K.\u00a0Ajith Kumar, and Christine Prasad. 2006. An Australasian Study of Reading and Comprehension Skills in Novice Programmers, Using the Bloom and SOLO Taxonomies. In Proceedings of the 8th Australasian Conference on Computing Education - Volume 52 (Hobart, Australia) (ACE \u201906). Australian Computer Society, Inc., AUS, 243\u2013252."},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1080\/08993408.2019.1565235"}],"event":{"name":"CSERC '21: The 10th Computer Science Education Research Conference","location":"Virtual Event Netherlands","acronym":"CSERC '21"},"container-title":["The 10th Computer Science Education Research Conference"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3507923.3507956","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3507923.3507956","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T18:10:17Z","timestamp":1750183817000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3507923.3507956"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,22]]},"references-count":35,"alternative-id":["10.1145\/3507923.3507956","10.1145\/3507923"],"URL":"https:\/\/doi.org\/10.1145\/3507923.3507956","relation":{},"subject":[],"published":{"date-parts":[[2021,11,22]]}}}