{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,24]],"date-time":"2026-01-24T20:13:43Z","timestamp":1769285623228,"version":"3.49.0"},"reference-count":67,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2023,3,14]],"date-time":"2023-03-14T00:00:00Z","timestamp":1678752000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Educ."],"published-print":{"date-parts":[[2023,6,30]]},"abstract":"<jats:p>To provide practice and assessment of computational thinking, we need specific problems students can solve. There are many such problems, but they are hard to find. Learning environments and assessments often use only specific types of problems and thus do not cover computational thinking in its whole scope. We provide an extensive catalog of well-structured computational thinking problem sets together with a systematic encoding of their features. Based on this encoding, we propose a four-level taxonomy that provides an organization of a wide variety of problems. The catalog, taxonomy, and problem features are useful for content authors, designers of learning environments, and researchers studying computational thinking.<\/jats:p>","DOI":"10.1145\/3578269","type":"journal-article","created":{"date-parts":[[2022,12,22]],"date-time":"2022-12-22T12:48:53Z","timestamp":1671713333000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["The Landscape of Computational Thinking Problems for Practice and Assessment"],"prefix":"10.1145","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8877-4729","authenticated-orcid":false,"given":"Radek","family":"Pel\u00e1nek","sequence":"first","affiliation":[{"name":"Masaryk University, Czech Republic"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5601-926X","authenticated-orcid":false,"given":"Tom\u00e1\u0161","family":"Effenberger","sequence":"additional","affiliation":[{"name":"Masaryk University, Czech Republic"}]}],"member":"320","published-online":{"date-parts":[[2023,3,14]]},"reference":[{"issue":"3","key":"e_1_3_3_2_2","first-page":"47","article-title":"A K-6 computational thinking curriculum framework: Implications for teacher knowledge","volume":"19","author":"Angeli Charoula","year":"2016","unstructured":"Charoula Angeli, Joke Voogt, Andrew Fluck, Mary Webb, Margaret Cox, Joyce Malyn-Smith, and Jason Zagami. 2016. A K-6 computational thinking curriculum framework: Implications for teacher knowledge. J. Educ. Technol. Soc. 19, 3 (2016), 47\u201357.","journal-title":"J. Educ. Technol. Soc."},{"key":"e_1_3_3_3_2","first-page":"1","volume-title":"American Association for Artificial Intelligence Educational Data Mining Workshop","author":"Barnes Tiffany","year":"2005","unstructured":"Tiffany Barnes. 2005. The q-matrix method: Mining student response data for knowledge. In American Association for Artificial Intelligence Educational Data Mining Workshop. AAAI Press, Washington, DC, 1\u20138."},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/1929887.1929905"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3015455"},{"key":"e_1_3_3_6_2","first-page":"595","volume-title":"International Conference on Artificial Intelligence in Education","author":"Boro\u0161 Petr","year":"2013","unstructured":"Petr Boro\u0161, Juraj Ni\u017enan, Radek Pel\u00e1nek, and Ji\u0159\u00ed \u0158ih\u00e1k. 2013. Automatic detection of concepts from problem solving times. In International Conference on Artificial Intelligence in Education. Springer, 595\u2013598."},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/1822348.1822352"},{"key":"e_1_3_3_8_2","first-page":"25","volume-title":"Proceedings of the Annual Meeting of the American Educational Research Association","volume":"1","author":"Brennan Karen","year":"2012","unstructured":"Karen Brennan and Mitchel Resnick. 2012. New frameworks for studying and assessing the development of computational thinking. In Proceedings of the Annual Meeting of the American Educational Research Association, Vol. 1. 25."},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3017680.3017765"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.tsc.2021.100888"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.97.3.404"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.28945\/2263"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-69924-8_2"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46747-4_3"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2015.00348"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2020.103832"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/11740.001.0001"},{"key":"e_1_3_3_18_2","unstructured":"Ruwan Devasurendra. 2020. Bebras Australia Computational Thinking Challenge."},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1080\/07468342.2018.1501989"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/2462476.2466518"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.5040\/9781350057142.ch-003"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.1080\/08993408.2021.1903248"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4471-6630-6"},{"key":"e_1_3_3_24_2","first-page":"1","article-title":"Computerized adaptive assessment of understanding of programming concepts in primary school children","author":"Hogenboom Sally A. M.","year":"2021","unstructured":"Sally A. M. Hogenboom, Felienne F. J. Hermans, and Han L. J. Van der Maas. 2021. Computerized adaptive assessment of understanding of programming concepts in primary school children. Comput. Sci. Educ. (2021), 1\u201330.","journal-title":"Comput. Sci. Educ."},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46747-4_13"},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2018.07.004"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/CIG.2019.8848107"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.3758\/s13428-019-01312-3"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/1930464.1930480"},{"key":"e_1_3_3_30_2","volume-title":"Proceedings of the 24th International FLAIRS Conference","author":"Jaru\u0161ek Petr","year":"2011","unstructured":"Petr Jaru\u0161ek and Radek Pel\u00e1nek. 2011. What determines difficulty of transport puzzles? In Proceedings of the 24th International FLAIRS Conference."},{"issue":"3","key":"e_1_3_3_31_2","first-page":"583","article-title":"A framework for computational thinking based on a systematic research review","volume":"4","author":"Kalelioglu Filiz","year":"2016","unstructured":"Filiz Kalelioglu, Yasemin G\u00fclbahar, and Volkan Kukul. 2016. A framework for computational thinking based on a systematic research review. Balt. J. Mod. Comput. 4, 3 (2016), 583.","journal-title":"Balt. J. Mod. Comput."},{"key":"e_1_3_3_32_2","unstructured":"K. Schulz S. Hobson and J. Zagami. 2016. Bebras Australia computational thinking challenge - tasks and solution 2016. Australia: Digital Careers Brisbane. https:\/\/www.csiro.au\/-\/media\/Digital-Careers\/Files\/Bebras-Files\/2016-Bebras-Solution-Guide-AU.pdf."},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/1089733.1089734"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cola.2021.101075"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1037\/h0093502"},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3293881.3295779"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/BB48857.2019.8941203"},{"key":"e_1_3_3_38_2","first-page":"223","volume-title":"Aptitude, Learning, and Instruction","author":"Malone Thomas W.","year":"2021","unstructured":"Thomas W. Malone and Mark R. Lepper. 2021. Making learning fun: A taxonomy of intrinsic motivations for learning. In Aptitude, Learning, and Instruction. Routledge, 223\u2013254."},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/2713609.2713610"},{"issue":"46","key":"e_1_3_3_40_2","first-page":"1","article-title":"Dr. scratch: Automatic analysis of scratch projects to assess and foster computational thinking","author":"Moreno-Le\u00f3n Jes\u00fas","year":"2015","unstructured":"Jes\u00fas Moreno-Le\u00f3n, Gregorio Robles, and Marcos Rom\u00e1n-Gonz\u00e1lez. 2015. Dr. scratch: Automatic analysis of scratch projects to assess and foster computational thinking. Rev. Educ. Dist. 46 (2015), 1\u201323.","journal-title":"Rev. Educ. Dist."},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3105726.3106178"},{"key":"e_1_3_3_42_2","volume-title":"Mindstorms: Children, Computers, and Powerful Ideas","author":"Papert Seymour A.","year":"2020","unstructured":"Seymour A. Papert. 2020. Mindstorms: Children, Computers, and Powerful Ideas. Basic Books."},{"key":"e_1_3_3_43_2","first-page":"157","volume-title":"Proceedings of the 8th Australasian Conference on Computing Education","author":"Parsons Dale","year":"2006","unstructured":"Dale Parsons and Patricia Haden. 2006. Parson\u2019s programming puzzles: A fun and effective learning tool for first programming courses. In Proceedings of the 8th Australasian Conference on Computing Education, Volume 52. 157\u2013163."},{"key":"e_1_3_3_44_2","article-title":"Karel the robot","author":"Pattis Richard","year":"1981","unstructured":"Richard Pattis, J. Roberts, and M. Stehlik. 1981. Karel the robot. In A Gentle Introduction to the Art of Programming.","journal-title":"A Gentle Introduction to the Art of Programming"},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-017-9193-2"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/TLT.2020.3027050"},{"key":"e_1_3_3_47_2","article-title":"Adaptive, intelligent, and personalized: Navigating the terminological maze behind educational technology","author":"Pel\u00e1nek Radek","year":"2021","unstructured":"Radek Pel\u00e1nek. 2021. Adaptive, intelligent, and personalized: Navigating the terminological maze behind educational technology. Int. J. Artif. Intell. Educ. 32, 1 (2021), 151--173.","journal-title":"Int. J. Artif. Intell. Educ."},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.1080\/08993408.2020.1832813"},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3077618"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF02300540"},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-13-6528-7_6"},{"key":"e_1_3_3_52_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2016.08.047"},{"key":"e_1_3_3_53_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2021.106707"},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1145\/2493394.2493403"},{"key":"e_1_3_3_55_2","doi-asserted-by":"publisher","DOI":"10.1080\/08993408.2019.1608781"},{"key":"e_1_3_3_56_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.edurev.2017.09.003"},{"key":"e_1_3_3_57_2","first-page":"13","volume-title":"CP Workshop on Modeling and Reformulating Constraint Satisfaction Problems","author":"Simonis Helmut","year":"2005","unstructured":"Helmut Simonis. 2005. Sudoku as a constraint problem. In CP Workshop on Modeling and Reformulating Constraint Satisfaction Problems, Vol. 12. 13\u201327."},{"key":"e_1_3_3_58_2","volume-title":"What Is the Name of This Book?","author":"Smullyan Raymond","year":"1986","unstructured":"Raymond Smullyan. 1986. What Is the Name of This Book? Touchstone Books."},{"issue":"4","key":"e_1_3_3_59_2","first-page":"2253","article-title":"Virtual laboratory for line follower robot competition","volume":"7","author":"Suwasono Suwasono","year":"2017","unstructured":"Suwasono Suwasono, Dwi Prihanto, Irawan Dwi Wahyono, and Andrew Nafalski. 2017. Virtual laboratory for line follower robot competition. Int. J. Electr. Comput. Eng. 7, 4 (2017), 2253.","journal-title":"Int. J. Electr. Comput. Eng."},{"key":"e_1_3_3_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/3230977.3230995"},{"key":"e_1_3_3_61_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2019.103798"},{"key":"e_1_3_3_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/2999541.2999542"},{"key":"e_1_3_3_63_2","doi-asserted-by":"publisher","DOI":"10.1080\/17470216008416717"},{"key":"e_1_3_3_64_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10956-015-9581-5"},{"key":"e_1_3_3_65_2","doi-asserted-by":"publisher","DOI":"10.1201\/9780429330704"},{"key":"e_1_3_3_66_2","doi-asserted-by":"publisher","DOI":"10.1145\/1118178.1118215"},{"key":"e_1_3_3_67_2","first-page":"1","article-title":"A systematic review of \u201cHour of Code\u201d research","author":"Yauney Jessica","year":"2022","unstructured":"Jessica Yauney, Scott R. Bartholomew, and Peter Rich. 2022. A systematic review of \u201cHour of Code\u201d research. Comput. Sci. Educ. (2022), 1\u201333.","journal-title":"Comput. Sci. Educ."},{"key":"e_1_3_3_68_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2019.103607"}],"container-title":["ACM Transactions on Computing Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3578269","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3578269","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:49:21Z","timestamp":1750182561000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3578269"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,14]]},"references-count":67,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,6,30]]}},"alternative-id":["10.1145\/3578269"],"URL":"https:\/\/doi.org\/10.1145\/3578269","relation":{},"ISSN":["1946-6226","1946-6226"],"issn-type":[{"value":"1946-6226","type":"print"},{"value":"1946-6226","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,14]]},"assertion":[{"value":"2022-01-28","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-12-09","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-03-14","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}