{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,14]],"date-time":"2026-01-14T16:57:18Z","timestamp":1768409838319,"version":"3.49.0"},"publisher-location":"New York, NY, USA","reference-count":66,"publisher":"ACM","license":[{"start":{"date-parts":[[2023,8,7]],"date-time":"2023-08-07T00:00:00Z","timestamp":1691366400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["21-21424"],"award-info":[{"award-number":["21-21424"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,8,7]]},"DOI":"10.1145\/3568813.3600140","type":"proceedings-article","created":{"date-parts":[[2023,9,10]],"date-time":"2023-09-10T15:33:07Z","timestamp":1694359987000},"page":"329-343","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Evaluating Beacons, the Role of Variables, Tracing, and Abstract Tracing for Teaching Novices to Understand Program Intent"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3473-9730","authenticated-orcid":false,"given":"Mohammed","family":"Hassan","sequence":"first","affiliation":[{"name":"University of Illinois, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9702-2796","authenticated-orcid":false,"given":"Kathryn","family":"Cunningham","sequence":"additional","affiliation":[{"name":"University of Illinois, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4601-4398","authenticated-orcid":false,"given":"Craig","family":"Zilles","sequence":"additional","affiliation":[{"name":"University of Illinois, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,9,10]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/1026487.1008013"},{"key":"e_1_3_2_1_2_1","volume-title":"BlueJ visual debugger for learning the execution of object-oriented programs?ACM Transactions on Computing Education (TOCE) 10, 2","author":"Bennedsen Jens","year":"2010","unstructured":"Jens Bennedsen and Carsten Schulte . 2010. BlueJ visual debugger for learning the execution of object-oriented programs?ACM Transactions on Computing Education (TOCE) 10, 2 ( 2010 ), 1\u201322. Jens Bennedsen and Carsten Schulte. 2010. BlueJ visual debugger for learning the execution of object-oriented programs?ACM Transactions on Computing Education (TOCE) 10, 2 (2010), 1\u201322."},{"key":"e_1_3_2_1_3_1","unstructured":"Sarah Blankenship. 2022. Learning to Reason About Code with Assertions: An Exploration with Two Student Populations. (2022).  Sarah Blankenship. 2022. Learning to Reason About Code with Assertions: An Exploration with Two Student Populations. (2022)."},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-7373(83)80031-5"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2015.36"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1124706.1121470"},{"key":"e_1_3_2_1_7_1","volume-title":"Metacognition and cognitive monitoring: A new area of cognitive\u2013developmental inquiry.American psychologist 34, 10","author":"Flavell H","year":"1979","unstructured":"John\u00a0 H Flavell . 1979. Metacognition and cognitive monitoring: A new area of cognitive\u2013developmental inquiry.American psychologist 34, 10 ( 1979 ), 906. John\u00a0H Flavell. 1979. Metacognition and cognitive monitoring: A new area of cognitive\u2013developmental inquiry.American psychologist 34, 10 (1979), 906."},{"key":"e_1_3_2_1_8_1","volume-title":"Tool-Aided Learning of Code Reasoning with Abstraction in the CS Curriculum. Informatics in Education 20, 4","author":"Fowler Megan","year":"2021","unstructured":"Megan Fowler , Jason Hallstrom , Joseph Hollingsworth , Eileen Kraemer , Murali Sitaraman , Yu-Shan Sun , Jiadi Wang , and Gloria Washington . 2021. Tool-Aided Learning of Code Reasoning with Abstraction in the CS Curriculum. Informatics in Education 20, 4 ( 2021 ). Megan Fowler, Jason Hallstrom, Joseph Hollingsworth, Eileen Kraemer, Murali Sitaraman, Yu-Shan Sun, Jiadi Wang, and Gloria Washington. 2021. Tool-Aided Learning of Code Reasoning with Abstraction in the CS Curriculum. Informatics in Education 20, 4 (2021)."},{"key":"e_1_3_2_1_9_1","volume-title":"Reevaluating the relationship between explaining, tracing, and writing skills in CS1 in a replication study. Computer Science Education","author":"Fowler Max","year":"2022","unstructured":"Max Fowler , David\u00a0 H Smith\u00a0IV, Mohammed Hassan , Seth Poulsen , Matthew West , and Craig Zilles . 2022. Reevaluating the relationship between explaining, tracing, and writing skills in CS1 in a replication study. Computer Science Education ( 2022 ), 1\u201329. Max Fowler, David\u00a0H Smith\u00a0IV, Mohammed Hassan, Seth Poulsen, Matthew West, and Craig Zilles. 2022. Reevaluating the relationship between explaining, tracing, and writing skills in CS1 in a replication study. Computer Science Education (2022), 1\u201329."},{"key":"e_1_3_2_1_10_1","volume-title":"Empirical studies of programmers: Fourth workshop","author":"Gellenbeck M","unstructured":"Edward\u00a0 M Gellenbeck and Curtis\u00a0 R Cook . 1991. An investigation of procedure and variable names as beacons during program comprehension . In Empirical studies of programmers: Fourth workshop . Ablex Publishing , Norwood, NJ , 65\u201381. Edward\u00a0M Gellenbeck and Curtis\u00a0R Cook. 1991. An investigation of procedure and variable names as beacons during program comprehension. In Empirical studies of programmers: Fourth workshop. Ablex Publishing, Norwood, NJ, 65\u201381."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/366413.364530"},{"key":"e_1_3_2_1_12_1","volume-title":"The role of gesture in communication and thinking. Trends in cognitive sciences 3, 11","author":"Goldin-Meadow Susan","year":"1999","unstructured":"Susan Goldin-Meadow . 1999. The role of gesture in communication and thinking. Trends in cognitive sciences 3, 11 ( 1999 ), 419\u2013429. Susan Goldin-Meadow. 1999. The role of gesture in communication and thinking. Trends in cognitive sciences 3, 11 (1999), 419\u2013429."},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.5555\/3288647.3288712"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/22627.22367"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3545945.3569741"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3408877.3432442"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3059009.3059036"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3344429.3372501"},{"key":"e_1_3_2_1_19_1","first-page":"1","article-title":"Through (tracking) their eyes: Abstraction and complexity in program comprehension","volume":"22","author":"Kather Philipp","year":"2021","unstructured":"Philipp Kather , Rodrigo Duran , and Jan Vahrenhold . 2021 . Through (tracking) their eyes: Abstraction and complexity in program comprehension . ACM Transactions on Computing Education (TOCE) 22 , 2 (2021), 1 \u2013 33 . Philipp Kather, Rodrigo Duran, and Jan Vahrenhold. 2021. Through (tracking) their eyes: Abstraction and complexity in program comprehension. ACM Transactions on Computing Education (TOCE) 22, 2 (2021), 1\u201333.","journal-title":"ACM Transactions on Computing Education (TOCE)"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1007996.1008014"},{"key":"e_1_3_2_1_21_1","volume-title":"Cognitive Processes in Program Comprehension: First Workshop. E. Soloway and S. Iyengar eds.","author":"Letovsky S","year":"1986","unstructured":"S Letovsky . 1986 . Cognitive Processes in Program Comprehension: First Workshop. E. Soloway and S. Iyengar eds. S Letovsky. 1986. Cognitive Processes in Program Comprehension: First Workshop. E. Soloway and S. Iyengar eds."},{"key":"e_1_3_2_1_22_1","volume-title":"The neglected middle novice programmer: Reading and writing without abstracting. National Advisory Committee on Computing Qualifications","author":"Lister RF","year":"2007","unstructured":"RF Lister . 2007. The neglected middle novice programmer: Reading and writing without abstracting. National Advisory Committee on Computing Qualifications ( 2007 ). RF Lister. 2007. The neglected middle novice programmer: Reading and writing without abstracting. National Advisory Committee on Computing Qualifications (2007)."},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1562877.1562930"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1016\/0164-1212(87)90033-1"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1404520.1404531"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.3758\/BF03334970"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3105726.3106168"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1186\/s40594-020-00222-7"},{"key":"e_1_3_2_1_29_1","volume-title":"Multi-institutional Study of Assessment of Programming Skills of First-year CS Students","author":"Michael","unstructured":"Michael McCracken 2001. A Multi-national , Multi-institutional Study of Assessment of Programming Skills of First-year CS Students . In Working Group Reports from ITiCSE on Innovation and Technology in Computer Science Education (Canterbury, UK) (ITiCSE-WGR \u201901). ACM, New York, NY, USA, 125\u2013180. Michael McCracken 2001. A Multi-national, Multi-institutional Study of Assessment of Programming Skills of First-year CS Students. In Working Group Reports from ITiCSE on Innovation and Technology in Computer Science Education (Canterbury, UK) (ITiCSE-WGR \u201901). ACM, New York, NY, USA, 125\u2013180."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3105726.3106178"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.5555\/1075040.1075044"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.5555\/54968.54975"},{"key":"e_1_3_2_1_33_1","volume-title":"Stimulus structures and mental representations in expert comprehension of computer programs. Cognitive psychology 19, 3","author":"Pennington Nancy","year":"1987","unstructured":"Nancy Pennington . 1987. Stimulus structures and mental representations in expert comprehension of computer programs. Cognitive psychology 19, 3 ( 1987 ), 295\u2013341. Nancy Pennington. 1987. Stimulus structures and mental representations in expert comprehension of computer programs. Cognitive psychology 19, 3 (1987), 295\u2013341."},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.4324\/9781003154730"},{"key":"e_1_3_2_1_35_1","volume-title":"The logic of scientific discovery","author":"Popper Karl","unstructured":"Karl Popper . 2005. The logic of scientific discovery . Routledge . Karl Popper. 2005. The logic of scientific discovery. Routledge."},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3230977.3230981"},{"key":"e_1_3_2_1_37_1","volume-title":"Proceedings of the Seventh Baltic Sea Conference on Computing Education Research (Koli Calling","author":"Raadt Michael","year":"2007","unstructured":"Michael Raadt , Mark Toleman , and Richard Watson . 2007 . Incorporating programming strategies explicitly into curricula . In Proceedings of the Seventh Baltic Sea Conference on Computing Education Research (Koli Calling 2007). Michael Raadt, Mark Toleman, and Richard Watson. 2007. Incorporating programming strategies explicitly into curricula. In Proceedings of the Seventh Baltic Sea Conference on Computing Education Research (Koli Calling 2007)."},{"key":"e_1_3_2_1_38_1","volume-title":"Proceedings of the Annual Meeting of the Cognitive Science Society, Vol.\u00a027","author":"Ritt Nikolaus","year":"2005","unstructured":"Nikolaus Ritt . 2005 . Analogy and Transfer: Encoding the Problem at the Right Level of Abstraction . In Proceedings of the Annual Meeting of the Cognitive Science Society, Vol.\u00a027 . Nikolaus Ritt. 2005. Analogy and Transfer: Encoding the Problem at the Right Level of Abstraction. In Proceedings of the Annual Meeting of the Cognitive Science Society, Vol.\u00a027."},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/HCC.2002.1046340"},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1080\/08993400600874584"},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/774833.774835"},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1080\/08993400500056563"},{"key":"e_1_3_2_1_43_1","volume-title":"Analogical transfer as guided by an abstraction process: The case of learning by doing in text editing.Journal of experimental psychology: learning, memory, and cognition 23, 6","author":"Sander Emmanuel","year":"1997","unstructured":"Emmanuel Sander and Jean-Fran\u00e7ois Richard . 1997. Analogical transfer as guided by an abstraction process: The case of learning by doing in text editing.Journal of experimental psychology: learning, memory, and cognition 23, 6 ( 1997 ), 1459. Emmanuel Sander and Jean-Fran\u00e7ois Richard. 1997. Analogical transfer as guided by an abstraction process: The case of learning by doing in text editing.Journal of experimental psychology: learning, memory, and cognition 23, 6 (1997), 1459."},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/1404520.1404535"},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1002\/tea.3660301011"},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.3390\/info12100424"},{"key":"e_1_3_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.5555\/1862712.1862734"},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/6592.6594"},{"key":"e_1_3_2_1_49_1","volume-title":"Knowledge and processes in the comprehension of computer programs. The nature of expertise","author":"Soloway Elliot","year":"1988","unstructured":"Elliot Soloway , Beth Adelson , and Kate Ehrlich . 1988. Knowledge and processes in the comprehension of computer programs. The nature of expertise ( 1988 ), 129\u2013152. Elliot Soloway, Beth Adelson, and Kate Ehrlich. 1988. Knowledge and processes in the comprehension of computer programs. The nature of expertise (1988), 129\u2013152."},{"key":"e_1_3_2_1_50_1","volume-title":"Visual program simulation in introductory programming education","author":"Juha Sorva","unstructured":"Juha Sorva 2012. Visual program simulation in introductory programming education . Aalto University . Juha Sorva 2012. Visual program simulation in introductory programming education. Aalto University."},{"key":"e_1_3_2_1_51_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2490822","article-title":"A review of generic program visualization systems for introductory programming education","volume":"13","author":"Sorva Juha","year":"2013","unstructured":"Juha Sorva , Ville Karavirta , and Lauri Malmi . 2013 . A review of generic program visualization systems for introductory programming education . ACM Transactions on Computing Education (TOCE) 13 , 4 (2013), 1 \u2013 64 . Juha Sorva, Ville Karavirta, and Lauri Malmi. 2013. A review of generic program visualization systems for introductory programming education. ACM Transactions on Computing Education (TOCE) 13, 4 (2013), 1\u201364.","journal-title":"ACM Transactions on Computing Education (TOCE)"},{"key":"e_1_3_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/6138.6145"},{"key":"e_1_3_2_1_53_1","volume-title":"A goal\/plan analysis of buggy Pascal programs. Human\u2013Computer Interaction 1, 2","author":"Spohrer C","year":"1985","unstructured":"James\u00a0 C Spohrer , Elliot Soloway , and Edgar Pope . 1985. A goal\/plan analysis of buggy Pascal programs. Human\u2013Computer Interaction 1, 2 ( 1985 ), 163\u2013207. James\u00a0C Spohrer, Elliot Soloway, and Edgar Pope. 1985. A goal\/plan analysis of buggy Pascal programs. Human\u2013Computer Interaction 1, 2 (1985), 163\u2013207."},{"key":"e_1_3_2_1_54_1","volume-title":"Neo-Piagetian theory and the novice programmer. Diss","author":"Teague Donna","year":"2015","unstructured":"Donna Teague . 2015. Neo-Piagetian theory and the novice programmer. Diss . Queensland University of Technology ( 2015 ). Donna Teague. 2015. Neo-Piagetian theory and the novice programmer. Diss. Queensland University of Technology (2015)."},{"key":"e_1_3_2_1_55_1","volume-title":"Proceedings of the 15th Australasian Computing Education Conference [Conferences in Research and Practice in Information Technology","volume":"95","author":"Teague Donna","year":"2013","unstructured":"Donna Teague , Malcolm Corney , Alireza Ahadi , and Raymond Lister . 2013 . A qualitative think aloud study of the early neo-piagetian stages of reasoning in novice programmers . In Proceedings of the 15th Australasian Computing Education Conference [Conferences in Research and Practice in Information Technology , Volume 136]. Australian Computer Society, 87\u2013 95 . Donna Teague, Malcolm Corney, Alireza Ahadi, and Raymond Lister. 2013. A qualitative think aloud study of the early neo-piagetian stages of reasoning in novice programmers. In Proceedings of the 15th Australasian Computing Education Conference [Conferences in Research and Practice in Information Technology, Volume 136]. Australian Computer Society, 87\u201395."},{"key":"e_1_3_2_1_56_1","unstructured":"Donna Teague and Raymond Lister. 2014. Blinded by their Plight: Tracing and the Preoperational Programmer.. In PPIG. 8.  Donna Teague and Raymond Lister. 2014. Blinded by their Plight: Tracing and the Preoperational Programmer.. In PPIG. 8."},{"key":"e_1_3_2_1_57_1","volume-title":"Conferences in Research and Practice in Information Technology Series.","author":"Teague Donna","year":"2014","unstructured":"Donna Teague and Raymond Lister . 2014 . Longitudinal think aloud study of a novice programmer . In Conferences in Research and Practice in Information Technology Series. Donna Teague and Raymond Lister. 2014. Longitudinal think aloud study of a novice programmer. In Conferences in Research and Practice in Information Technology Series."},{"key":"e_1_3_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/1269900.1268853"},{"key":"e_1_3_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-7373(85)80054-7"},{"key":"e_1_3_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1109\/2.402076"},{"key":"e_1_3_2_1_61_1","volume-title":"Unraveling novices","author":"Weeda Renske","year":"2023","unstructured":"Renske Weeda , Sjaak Smetsers , and Erik Barendsen . 2023. Unraveling novices \u2019 code composition difficulties. Computer Science Education ( 2023 ), 1\u201328. Renske Weeda, Sjaak Smetsers, and Erik Barendsen. 2023. Unraveling novices\u2019 code composition difficulties. Computer Science Education (2023), 1\u201328."},{"key":"e_1_3_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1984.5010248"},{"key":"e_1_3_2_1_63_1","unstructured":"JL Whalley Tony Clear Phil Robbins and Errol Thompson. 2011. Salient elements in novice solutions to code writing problems. (2011).  JL Whalley Tony Clear Phil Robbins and Errol Thompson. 2011. Salient elements in novice solutions to code writing problems. (2011)."},{"key":"e_1_3_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-7373(05)80090-2"},{"key":"e_1_3_2_1_65_1","volume-title":"Proceedings of the third international conference on human-computer interaction on Designing and using human-computer interfaces and knowledge based systems (2nd ed.). 82\u201387","author":"Wiedenbeck Susan","year":"1989","unstructured":"Susan Wiedenbeck and Jean Scholtz . 1989 . Beacons: A knowledge structure in program comprehension . In Proceedings of the third international conference on human-computer interaction on Designing and using human-computer interfaces and knowledge based systems (2nd ed.). 82\u201387 . Susan Wiedenbeck and Jean Scholtz. 1989. Beacons: A knowledge structure in program comprehension. In Proceedings of the third international conference on human-computer interaction on Designing and using human-computer interfaces and knowledge based systems (2nd ed.). 82\u201387."},{"key":"e_1_3_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1080\/08993408.2019.1565235"}],"event":{"name":"ICER 2023: ACM Conference on International Computing Education Research","location":"Chicago IL USA","acronym":"ICER 2023"},"container-title":["Proceedings of the 2023 ACM Conference on International Computing Education Research V.1"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3568813.3600140","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3568813.3600140","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3568813.3600140","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:49:08Z","timestamp":1750182548000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3568813.3600140"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,7]]},"references-count":66,"alternative-id":["10.1145\/3568813.3600140","10.1145\/3568813"],"URL":"https:\/\/doi.org\/10.1145\/3568813.3600140","relation":{},"subject":[],"published":{"date-parts":[[2023,8,7]]},"assertion":[{"value":"2023-09-10","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}