{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T06:53:24Z","timestamp":1777100004444,"version":"3.51.4"},"reference-count":28,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2004,6,28]],"date-time":"2004-06-28T00:00:00Z","timestamp":1088380800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGCSE Bull."],"published-print":{"date-parts":[[2004,12]]},"abstract":"<jats:p>A study by a ITiCSE 2001 working group (\"the McCracken Group\") established that many students do not know how to program at the conclusion of their introductory courses. A popular explanation for this incapacity is that the students lack the ability to problem-solve. That is, they lack the ability to take a problem description, decompose it into sub-problems and implement them, then assemble the pieces into a complete solution. An alternative explanation is that many students have a fragile grasp of both basic programming principles and the ability to systematically carry out routine programming tasks, such as tracing (or \"desk checking\") through code. This ITiCSE 2004 working group studied the alternative explanation, by testing students from seven countries, in two ways. First, students were tested on their ability to predict the outcome of executing a short piece of code. Second, students were tested on their ability, when given the desired function of short piece of near-complete code, to select the correct completion of the code from a small set of possibilities. Many students were weak at these tasks, especially the latter task, suggesting that such students have a fragile grasp of skills that are a prerequisite for problem-solving.<\/jats:p>","DOI":"10.1145\/1041624.1041673","type":"journal-article","created":{"date-parts":[[2005,1,26]],"date-time":"2005-01-26T16:49:14Z","timestamp":1106758154000},"page":"119-150","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":257,"title":["A multi-national study of reading and tracing skills in novice programmers"],"prefix":"10.1145","volume":"36","author":[{"given":"Raymond","family":"Lister","sequence":"first","affiliation":[{"name":"University of Technology, Sydney Broadway, NSW, Australia"}]},{"given":"Elizabeth S.","family":"Adams","sequence":"additional","affiliation":[{"name":"James Madison University, Harrisonburg, VA"}]},{"given":"Sue","family":"Fitzgerald","sequence":"additional","affiliation":[{"name":"Metropolitan State University, St. Paul, MN"}]},{"given":"William","family":"Fone","sequence":"additional","affiliation":[{"name":"Staffordshire University, Stafford, ST, United Kingdom"}]},{"given":"John","family":"Hamer","sequence":"additional","affiliation":[{"name":"University of Auckland, Auckland, New Zealand"}]},{"given":"Morten","family":"Lindholm","sequence":"additional","affiliation":[{"name":"Aarhus University, Aarhus, Denmark"}]},{"given":"Robert","family":"McCartney","sequence":"additional","affiliation":[{"name":"University of Connecticut, Storrs, CT"}]},{"given":"Jan Erik","family":"Mostr\u00f6m","sequence":"additional","affiliation":[{"name":"Ume\u00e1 University, Ume\u00e1, Sweden"}]},{"given":"Kate","family":"Sanders","sequence":"additional","affiliation":[{"name":"Rhode Island College, Providence, RI"}]},{"given":"Otto","family":"Sepp\u00e4l\u00e4","sequence":"additional","affiliation":[{"name":"Helsinki University of Technology, TKK, Finland"}]},{"given":"Beth","family":"Simon","sequence":"additional","affiliation":[{"name":"University of San Diego, San Diego, CA"}]},{"given":"Lynda","family":"Thomas","sequence":"additional","affiliation":[{"name":"University of Wales, Aberystwyth"}]}],"member":"320","published-online":{"date-parts":[[2004,6,28]]},"reference":[{"key":"e_1_2_1_1_1","first-page":"543","volume-title":"International Journal of man-Machine Studies, 18","author":"Brooks R.","year":"1983","unstructured":"Brooks , R. ( 1983 ) Towards a theory of the comprehension of computer programs . International Journal of man-Machine Studies, 18 , pp. 543 -- 554 . Brooks, R. (1983) Towards a theory of the comprehension of computer programs. International Journal of man-Machine Studies, 18, pp. 543--554."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/299649.299673"},{"key":"e_1_2_1_3_1","first-page":"59","volume-title":"Gray, W, and Boehm-Davis, D. (Eds) Empirical Studies of Programmers: 6th Workshop","author":"Davies S.","year":"1996","unstructured":"Davies , S. ( 1996 ) Display-based problem solving strategies in computer programming . In Gray, W, and Boehm-Davis, D. (Eds) Empirical Studies of Programmers: 6th Workshop . Ablex Publishing Corporation , NJ. pp. 59 -- 76 . Davies, S. (1996) Display-based problem solving strategies in computer programming. In Gray, W, and Boehm-Davis, D. (Eds) Empirical Studies of Programmers: 6th Workshop. Ablex Publishing Corporation, NJ. pp. 59--76."},{"key":"e_1_2_1_4_1","first-page":"206","volume-title":"Hoc, J, Green, T, Samurcay, and Gilmore, D. (Eds) Psychology of Programming","author":"Detienne F.","year":"1990","unstructured":"Detienne , F. ( 1990 ) Expert Programming Knowledge: A Schema-based Approach . In Hoc, J, Green, T, Samurcay, and Gilmore, D. (Eds) Psychology of Programming . Academic Press , London . pp 206 -- 222 . Detienne, F. (1990) Expert Programming Knowledge: A Schema-based Approach. In Hoc, J, Green, T, Samurcay, and Gilmore, D. (Eds) Psychology of Programming. Academic Press, London. pp 206--222."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-7373(05)80122-1"},{"key":"e_1_2_1_6_1","doi-asserted-by":"crossref","DOI":"10.21236\/ADA228026","volume-title":"Reading Computer Programs: Instructor's Guide and Exercises Software Engineering Institute","author":"Deimel L. E.","year":"1990","unstructured":"Deimel , L. E. & Naveda , J. F. ( 1990 ) Reading Computer Programs: Instructor's Guide and Exercises Software Engineering Institute , Carnegie-Mellon University . http:\/\/www.deimel.org\/comp_sci\/reading_computer_programs.htm (August 2004) Updated bibliography at http:\/\/www2.umassd.edu\/SWPI\/Process Bibliography\/bib-codereading2.html (August 2004). Deimel, L. E. & Naveda, J. F. (1990) Reading Computer Programs: Instructor's Guide and Exercises Software Engineering Institute, Carnegie-Mellon University. http:\/\/www.deimel.org\/comp_sci\/reading_computer_programs.htm (August 2004) Updated bibliography at http:\/\/www2.umassd.edu\/SWPI\/Process Bibliography\/bib-codereading2.html (August 2004)."},{"key":"e_1_2_1_7_1","volume-title":"Some Difficulties of Learning to Program","author":"du Boulay B.","year":"1989","unstructured":"du Boulay , B. ( 1989 ) Some Difficulties of Learning to Program . In Soloway, E. and Spohrer, J., Eds. pp. 283--299. du Boulay, B. (1989) Some Difficulties of Learning to Program. In Soloway, E. and Spohrer, J., Eds. pp. 283--299."},{"key":"e_1_2_1_8_1","volume-title":"Essentials of Educational Measurement","author":"Ebel R.","year":"1986","unstructured":"Ebel , R. and Frisbie , D . ( 1986 ) Essentials of Educational Measurement . Prentice Hall , Englewood Cliffs, NJ . Ebel, R. and Frisbie, D. (1986) Essentials of Educational Measurement. Prentice Hall, Englewood Cliffs, NJ."},{"key":"e_1_2_1_9_1","volume-title":"Developing and Validating Multiple-Choice Questions","author":"Haladyna T.","year":"1999","unstructured":"Haladyna , T. ( 1999 ) Developing and Validating Multiple-Choice Questions ( 2 nd Edition), Lawrence Erlbaum Associates , Mahwah, NJ . Haladyna, T. (1999) Developing and Validating Multiple-Choice Questions (2nd Edition), Lawrence Erlbaum Associates, Mahwah, NJ.","edition":"2"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1007996.1008014"},{"key":"e_1_2_1_11_1","volume-title":"Measurement and Assessment in Teaching","author":"Linn R.","year":"1995","unstructured":"Linn , R. and Gronlund , N . ( 1995 ) Measurement and Assessment in Teaching , Prentice Hall , Upper Saddle River, NJ. Linn, R. and Gronlund, N. (1995) Measurement and Assessment in Teaching, Prentice Hall, Upper Saddle River, NJ."},{"key":"e_1_2_1_12_1","volume-title":"Availability of working group data","author":"Lister R.","year":"2004","unstructured":"Lister , R. ( 2004 ) Availability of working group data . http:\/\/www-staff.it.uts.edu.au\/~raymond\/leeds2004. Lister, R. (2004) Availability of working group data. http:\/\/www-staff.it.uts.edu.au\/~raymond\/leeds2004."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/572139.572181"},{"key":"e_1_2_1_14_1","first-page":"100","volume-title":"Olson, G., Sheppard, S., and Soloway, E. (Eds) Empirical Studies of Programmers: Second Workshop.","author":"Pennington N.","year":"1987","unstructured":"Pennington , N. ( 1987 ) Comprehension Strategies in Programming . In Olson, G., Sheppard, S., and Soloway, E. (Eds) Empirical Studies of Programmers: Second Workshop. Ablex, NJ, USA . pp 100 -- 113 . Pennington, N. (1987) Comprehension Strategies in Programming. In Olson, G., Sheppard, S., and Soloway, E. (Eds) Empirical Studies of Programmers: Second Workshop. Ablex, NJ, USA. pp 100--113."},{"key":"e_1_2_1_15_1","first-page":"213","volume-title":"Soloway, E. and Iyengar, S. (Eds)","author":"Perkins D.","year":"1986","unstructured":"Perkins , D. and Martin , F . ( 1986 ) Fragile Knowledge and Neglected Strategies in Novice Programmers . In Soloway, E. and Iyengar, S. (Eds) pp. 213 -- 229 . Perkins, D. and Martin, F. (1986) Fragile Knowledge and Neglected Strategies in Novice Programmers. In Soloway, E. and Iyengar, S. (Eds) pp. 213--229."},{"key":"e_1_2_1_16_1","volume-title":"Conditions of Learning in Novice Programmers","author":"Perkins D","year":"1989","unstructured":"Perkins , D , Hancock , C , Hobbs , R , Martin , F , and Simmons , R . ( 1989 ). Conditions of Learning in Novice Programmers . In Soloway, E. and Spohrer, J., Eds. pp 261--279. Perkins, D, Hancock, C, Hobbs, R, Martin, F, and Simmons, R. (1989). Conditions of Learning in Novice Programmers. In Soloway, E. and Spohrer, J., Eds. pp 261--279."},{"key":"e_1_2_1_17_1","volume-title":"Plans in Programming: Definition, Demonstration and Development","author":"Rist R. S.","year":"1986","unstructured":"Rist , R. S. ( 1986 ). Plans in Programming: Definition, Demonstration and Development . In Soloway, E. and Iyengar, S., Eds. pp 28--47. Rist, R. S. (1986). Plans in Programming: Definition, Demonstration and Development. In Soloway, E. and Iyengar, S., Eds. pp 28--47."},{"key":"e_1_2_1_18_1","volume-title":"Computer Science Education Research. Swets & Zeitlinger.","author":"Rist R.","year":"2004","unstructured":"Rist , R. ( 2004 ) Learning to Program: Schema Creation, Application, and Evaluation . In Fincher, S and Petre, M., Eds (2004) Computer Science Education Research. Swets & Zeitlinger. Rist, R. (2004) Learning to Program: Schema Creation, Application, and Evaluation. In Fincher, S and Petre, M., Eds (2004) Computer Science Education Research. Swets & Zeitlinger."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1984.5010283"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/6592.6594"},{"key":"e_1_2_1_21_1","volume-title":"Empirical Studies of Programmers","author":"Soloway E.","year":"1986","unstructured":"Soloway , E. and Iyengar , S. , Eds ( 1986 ) Empirical Studies of Programmers . Ablex, NJ, USA . Soloway, E. and Iyengar, S., Eds (1986) Empirical Studies of Programmers. Ablex, NJ, USA."},{"key":"e_1_2_1_22_1","first-page":"129","volume-title":"Glaser, M, Chi, R, Farr, M, Glaser, R (Eds) The Nature of Expertise","author":"Soloway E","year":"1988","unstructured":"Soloway , E , Adelson , B , and Ehrlich , K . ( 1988 ) Knowledge and Processes in the Comprehension of Computer Programs . In Glaser, M, Chi, R, Farr, M, Glaser, R (Eds) The Nature of Expertise . Lawrence Erlbaum Associates , Hillsdale, NJ, USA . pp 129 -- 152 . Soloway, E, Adelson, B, and Ehrlich, K. (1988) Knowledge and Processes in the Comprehension of Computer Programs. In Glaser, M, Chi, R, Farr, M, Glaser, R (Eds) The Nature of Expertise. Lawrence Erlbaum Associates, Hillsdale, NJ, USA. pp 129--152."},{"key":"e_1_2_1_23_1","volume-title":"Studying the Novice Programmer","author":"Soloway E.","year":"1989","unstructured":"Soloway , E. and and Spohrer , J , Eds ( 1989 ), Studying the Novice Programmer . Lawrence Erlbaum Associates , Hillsdale, NJ , 1989. Soloway, E. and and Spohrer, J, Eds (1989), Studying the Novice Programmer. Lawrence Erlbaum Associates, Hillsdale, NJ, 1989."},{"key":"e_1_2_1_24_1","first-page":"230","volume-title":"Soloway, E. and Iyengar, S. (Eds)","author":"Spohrer J.","year":"1986","unstructured":"Spohrer , J. and Soloway , E . ( 1986 ) Analyzing the High Frequency Bugs in Novice Programs . In Soloway, E. and Iyengar, S. (Eds) pp. 230 -- 251 . Spohrer, J. and Soloway, E. (1986) Analyzing the High Frequency Bugs in Novice Programs. In Soloway, E. and Iyengar, S. (Eds) pp. 230--251."},{"key":"e_1_2_1_25_1","volume-title":"Novice Mistakes: Are the Folk Wisdoms Correct? In Soloway","author":"Spohrer J.","year":"1989","unstructured":"Spohrer , J. and Soloway , E . ( 1989 ) Novice Mistakes: Are the Folk Wisdoms Correct? In Soloway , E. and Spohrer, J., Eds. pp 401--416. Spohrer, J. and Soloway, E. (1989) Novice Mistakes: Are the Folk Wisdoms Correct? In Soloway, E. and Spohrer, J., Eds. pp 401--416."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/971300.971390"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-7373(85)80041-9"},{"key":"e_1_2_1_28_1","volume-title":"Processes in Computer Program Comprehension","author":"Wiedenbeck S.","year":"1986","unstructured":"Wiedenbeck , S. ( 1986 ) Processes in Computer Program Comprehension . In Soloway, E. and Iyengar, S., Eds. pp 48--57. Wiedenbeck, S. (1986) Processes in Computer Program Comprehension. In Soloway, E. and Iyengar, S., Eds. pp 48--57."}],"container-title":["ACM SIGCSE Bulletin"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1041624.1041673","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1041624.1041673","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:43:42Z","timestamp":1750286622000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1041624.1041673"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2004,6,28]]},"references-count":28,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2004,12]]}},"alternative-id":["10.1145\/1041624.1041673"],"URL":"https:\/\/doi.org\/10.1145\/1041624.1041673","relation":{"is-identical-to":[{"id-type":"doi","id":"10.1145\/1044550.1041673","asserted-by":"subject"}]},"ISSN":["0097-8418"],"issn-type":[{"value":"0097-8418","type":"print"}],"subject":[],"published":{"date-parts":[[2004,6,28]]},"assertion":[{"value":"2004-06-28","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}