{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,11]],"date-time":"2026-06-11T07:43:36Z","timestamp":1781163816053,"version":"3.54.1"},"reference-count":28,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2004,6,28]],"date-time":"2004-06-28T00:00:00Z","timestamp":1088380800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGCSE Bull."],"published-print":{"date-parts":[[2004,12,1]]},"abstract":"<jats:p>A study by a ITiCSE 2001 working group (\"the McCracken Group\") established that many students do not know how to program at the conclusion of their introductory courses. A popular explanation for this incapacity is that the students lack the ability to problem-solve. That is, they lack the ability to take a problem description, decompose it into sub-problems and implement them, then assemble the pieces into a complete solution. An alternative explanation is that many students have a fragile grasp of both basic programming principles and the ability to systematically carry out routine programming tasks, such as tracing (or \"desk checking\") through code. This ITiCSE 2004 working group studied the alternative explanation, by testing students from seven countries, in two ways. First, students were tested on their ability to predict the outcome of executing a short piece of code. Second, students were tested on their ability, when given the desired function of short piece of near-complete code, to select the correct completion of the code from a small set of possibilities. Many students were weak at these tasks, especially the latter task, suggesting that such students have a fragile grasp of skills that are a prerequisite for problem-solving.<\/jats:p>","DOI":"10.1145\/1041624.1041673","type":"journal-article","created":{"date-parts":[[2005,1,26]],"date-time":"2005-01-26T11:49:14Z","timestamp":1106740154000},"page":"119-150","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":262,"title":["A multi-national study of reading and tracing skills in novice programmers"],"prefix":"10.1145","volume":"36","author":[{"given":"Raymond","family":"Lister","sequence":"first","affiliation":[{"name":"University of Technology, Sydney Broadway, NSW, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elizabeth S.","family":"Adams","sequence":"additional","affiliation":[{"name":"James Madison University, Harrisonburg, VA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sue","family":"Fitzgerald","sequence":"additional","affiliation":[{"name":"Metropolitan State University, St. Paul, MN"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"William","family":"Fone","sequence":"additional","affiliation":[{"name":"Staffordshire University, Stafford, ST, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"John","family":"Hamer","sequence":"additional","affiliation":[{"name":"University of Auckland, Auckland, New Zealand"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Morten","family":"Lindholm","sequence":"additional","affiliation":[{"name":"Aarhus University, Aarhus, Denmark"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robert","family":"McCartney","sequence":"additional","affiliation":[{"name":"University of Connecticut, Storrs, CT"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jan Erik","family":"Mostr\u00f6m","sequence":"additional","affiliation":[{"name":"Ume\u00e1 University, Ume\u00e1, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kate","family":"Sanders","sequence":"additional","affiliation":[{"name":"Rhode Island College, Providence, RI"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Otto","family":"Sepp\u00e4l\u00e4","sequence":"additional","affiliation":[{"name":"Helsinki University of Technology, TKK, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Beth","family":"Simon","sequence":"additional","affiliation":[{"name":"University of San Diego, San Diego, CA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lynda","family":"Thomas","sequence":"additional","affiliation":[{"name":"University of Wales, Aberystwyth"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2004,6,28]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-7373(83)80031-5"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/299649.299673"},{"key":"e_1_2_1_3_1","first-page":"59","volume-title":"Gray, W, and Boehm-Davis, D. (Eds) Empirical Studies of Programmers: 6th Workshop","author":"Davies S.","year":"1996","unstructured":"Davies, S. (1996) Display-based problem solving strategies in computer programming. In Gray, W, and Boehm-Davis, D. (Eds) Empirical Studies of Programmers: 6th Workshop. Ablex Publishing Corporation, NJ. pp. 59--76."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/B978-0-12-350772-3.50018-5"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-7373(05)80122-1"},{"key":"e_1_2_1_6_1","volume-title":"Reading Computer Programs: Instructor's Guide and Exercises Software Engineering Institute","author":"Deimel L. E.","year":"1990","unstructured":"Deimel, L. E. & Naveda, J. F. (1990) Reading Computer Programs: Instructor's Guide and Exercises Software Engineering Institute, Carnegie-Mellon University. http:\/\/www.deimel.org\/comp_sci\/reading_computer_programs.htm (August 2004) Updated bibliography at http:\/\/www2.umassd.edu\/SWPI\/Process Bibliography\/bib-codereading2.html (August 2004)."},{"key":"e_1_2_1_7_1","volume-title":"Some Difficulties of Learning to Program","author":"du Boulay B.","year":"1989","unstructured":"du Boulay, B. (1989) Some Difficulties of Learning to Program. In Soloway, E. and Spohrer, J., Eds. pp. 283--299."},{"key":"e_1_2_1_8_1","volume-title":"Essentials of Educational Measurement","author":"Ebel R.","year":"1986","unstructured":"Ebel, R. and Frisbie, D. (1986) Essentials of Educational Measurement. Prentice Hall, Englewood Cliffs, NJ."},{"key":"e_1_2_1_9_1","volume-title":"Developing and Validating Multiple-Choice Questions","author":"Haladyna T.","year":"1999","unstructured":"Haladyna, T. (1999) Developing and Validating Multiple-Choice Questions (2nd Edition), Lawrence Erlbaum Associates, Mahwah, NJ.","edition":"2"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1007996.1008014"},{"key":"e_1_2_1_11_1","volume-title":"Measurement and Assessment in Teaching","author":"Linn R.","year":"1995","unstructured":"Linn, R. and Gronlund, N. (1995) Measurement and Assessment in Teaching, Prentice Hall, Upper Saddle River, NJ."},{"key":"e_1_2_1_12_1","volume-title":"Availability of working group data","author":"Lister R.","year":"2004","unstructured":"Lister, R. (2004) Availability of working group data. http:\/\/www-staff.it.uts.edu.au\/~raymond\/leeds2004."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/572139.572181"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.5555\/54968.54975"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.5555\/21842.28896"},{"key":"e_1_2_1_16_1","volume-title":"Conditions of Learning in Novice Programmers","author":"Perkins D","year":"1989","unstructured":"Perkins, D, Hancock, C, Hobbs, R, Martin, F, and Simmons, R. (1989). Conditions of Learning in Novice Programmers. In Soloway, E. and Spohrer, J., Eds. pp 261--279."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.5555\/21842.28884"},{"key":"e_1_2_1_18_1","volume-title":"Computer Science Education Research. Swets & Zeitlinger.","author":"Rist R.","year":"2004","unstructured":"Rist, R. (2004) Learning to Program: Schema Creation, Application, and Evaluation. In Fincher, S and Petre, M., Eds (2004) Computer Science Education Research. Swets & Zeitlinger."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1984.5010283"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/6592.6594"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.5555\/576696"},{"key":"e_1_2_1_22_1","first-page":"129","volume-title":"Glaser, M, Chi, R, Farr, M, Glaser, R (Eds) The Nature of Expertise","author":"Soloway E","year":"1988","unstructured":"Soloway, E, Adelson, B, and Ehrlich, K. (1988) Knowledge and Processes in the Comprehension of Computer Programs. In Glaser, M, Chi, R, Farr, M, Glaser, R (Eds) The Nature of Expertise. Lawrence Erlbaum Associates, Hillsdale, NJ, USA. pp 129--152."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.5555\/576212"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.5555\/21842.28897"},{"key":"e_1_2_1_25_1","volume-title":"Novice Mistakes: Are the Folk Wisdoms Correct? In Soloway","author":"Spohrer J.","year":"1989","unstructured":"Spohrer, J. and Soloway, E. (1989) Novice Mistakes: Are the Folk Wisdoms Correct? In Soloway, E. and Spohrer, J., Eds. pp 401--416."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/971300.971390"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-7373(85)80041-9"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.5555\/21842.28885"}],"container-title":["ACM SIGCSE Bulletin"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1041624.1041673","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1041624.1041673","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T22:20:54Z","timestamp":1780438854000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1041624.1041673"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2004,6,28]]},"references-count":28,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2004,12,1]]}},"alternative-id":["10.1145\/1041624.1041673"],"URL":"https:\/\/doi.org\/10.1145\/1041624.1041673","relation":{"is-identical-to":[{"id-type":"doi","id":"10.1145\/1044550.1041673","asserted-by":"subject"}]},"ISSN":["0097-8418"],"issn-type":[{"value":"0097-8418","type":"print"}],"subject":[],"published":{"date-parts":[[2004,6,28]]},"assertion":[{"value":"2004-06-28","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}