{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,19]],"date-time":"2025-09-19T11:35:30Z","timestamp":1758281730145,"version":"3.44.0"},"reference-count":83,"publisher":"Association for Computing Machinery (ACM)","issue":"4","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Educ."],"published-print":{"date-parts":[[2025,12,31]]},"abstract":"<jats:p>The current insertion of Machine Learning (ML) in our everyday life demonstrates the importance of introducing the teaching of a basic understanding of ML already in school. Accompanying this trend arises the need to assess the students\u2019 learning of ML, yet so far only a few assessment models have been proposed, most of them rather simple, based on quizzes or self-assessments. In this context, we previously developed and evaluated a model for the performance-based assessment of the student\u2019s learning of ML for image classification based on the ML artifacts created by the students as a learning outcome as part of the \u201cML for All\u201d course, indicating good reliability (coefficient Omega\u2009 = \u20090.834) and validity. Evolving this assessment model, we present in this article a scale associated with the level of student learning and a descriptive interpretation of student performance using Item Response Theory (IRT) based on data collected from 240 students. The results of this research can support the learning of ML by providing feedback to students and supporting teachers.<\/jats:p>","DOI":"10.1145\/3749993","type":"journal-article","created":{"date-parts":[[2025,7,22]],"date-time":"2025-07-22T22:40:08Z","timestamp":1753224008000},"page":"1-30","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Scale for Assessing the Learning of Machine Learning for Image Classification in Middle and High School Using the \u201cML for All\u201d Course"],"prefix":"10.1145","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5653-7155","authenticated-orcid":false,"given":"Marcelo Fernando","family":"Rauber","sequence":"first","affiliation":[{"name":"Federal University of Santa Catarina, Florianopolis, Brazil and Instituto Federal Catarinense, Cambori\u00fa, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6566-1606","authenticated-orcid":false,"given":"Christiane Gresse von","family":"Wangenheim","sequence":"additional","affiliation":[{"name":"Federal University of Santa Catarina, Florianopolis, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0474-3640","authenticated-orcid":false,"given":"Pedro Alberto","family":"Barbetta","sequence":"additional","affiliation":[{"name":"Federal University of Santa Catarina, Florianopolis, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6280-2525","authenticated-orcid":false,"given":"Adriano Ferreti","family":"Borgatto","sequence":"additional","affiliation":[{"name":"Federal University of Santa Catarina, Florianopolis, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1952-0909","authenticated-orcid":false,"given":"Ramon Mayor","family":"Martins","sequence":"additional","affiliation":[{"name":"Federal University of Santa Catarina, Florianopolis, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6550-9092","authenticated-orcid":false,"given":"Jean Carlo Rossa","family":"Hauck","sequence":"additional","affiliation":[{"name":"Federal University of Santa Catarina, Florianopolis, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,9,18]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.15388\/infedu.2018.08"},{"issue":"3","key":"e_1_3_2_3_2","first-page":"479","article-title":"A proposal for performance-based assessment of the learning of machine learning concepts and practices in K-12","volume":"21","author":"von Wangenheim Christiane Gresse","year":"2021","unstructured":"Christiane Gresse von Wangenheim, Nathalia da Cruz Alves, Marcelo F. Rauber, Jean C. R. Hauck, and Ibrahim H. Yeter. 2021. A proposal for performance-based assessment of the learning of machine learning concepts and practices in K-12. Informatics in Education. 21, 3 (2021), 479\u2013500.","journal-title":"Informatics in Education"},{"key":"e_1_3_2_4_2","first-page":"1","article-title":"Machine learning for all\u2014Introducing machine learning in K-12","author":"von Wangenheim Christiane Gresse","year":"2020","unstructured":"Christiane Gresse von Wangenheim, L\u00edvia S. Marques, and Jean C. R. Hauck. 2020. Machine learning for all\u2014Introducing machine learning in K-12. SocArXiv 11, 1 (2020), 1\u201310.","journal-title":"SocArXiv"},{"key":"e_1_3_2_5_2","first-page":"1","article-title":"Machine learning for all! Introducing machine learning in middle and high school","author":"Martins R. M.","year":"2023","unstructured":"Martins R. M., von Wangenheim C. G., Rauber M. F., and Hauck J. C. 2023. Machine learning for all! Introducing machine learning in middle and high school. International Journal of Artificial Intelligence in Education 34, 2 (2023), 1\u201339.","journal-title":"International Journal of Artificial Intelligence in Education"},{"issue":"2","key":"e_1_3_2_6_2","first-page":"295","article-title":"Assessing the learning of machine learning in K-12: A ten-year systematic mapping","volume":"22","author":"Rauber M. F.","year":"2022","unstructured":"Rauber M. F. and Gresse von Wangenheim C. 2022. Assessing the learning of machine learning in K-12: A ten-year systematic mapping. Informatics in Education. 22, 2 (2022), 295\u2013328.","journal-title":"Informatics in Education"},{"issue":"2","key":"e_1_3_2_7_2","first-page":"409","article-title":"Reliability and validity of an automated model for assessing the learning of machine learning in middle and high school","volume":"23","author":"Rauber M. F.","year":"2023","unstructured":"Rauber M. F., Gresse von Wangenheim C., Barbetta P. A., Borgatto A. F., Martins R. M., and Hauck J. R. 2023. Reliability and validity of an automated model for assessing the learning of machine learning in middle and high school. Informatics in Education. 23, 2 (2023), 409\u2013437.","journal-title":"Informatics in Education"},{"key":"e_1_3_2_8_2","first-page":"1","volume-title":"Proceedings of Brazilian Symposium on Computer Education","author":"Alves Nathalia C.","year":"2021","unstructured":"Nathalia C. Alves, Christiane Gresse von Wangenheim, Jean C. R. Hauck, and Adriano F. Borgatto. 2021. An item response theory analysis of algorithms and programming concepts in app inventor projects. In Proceedings of Brazilian Symposium on Computer Education, 1\u201311."},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3328778.3366840"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1177\/0013164417713570"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP.2019.00042"},{"key":"e_1_3_2_12_2","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-58689-2","volume-title":"Advancing Human Assessment: The Methodological, Psychological and Policy Contributions of ETS","author":"Bennett Randy E.","year":"2017","unstructured":"Randy E. Bennett and Matthias von Davier (Eds.). 2017. Advancing Human Assessment: The Methodological, Psychological and Policy Contributions of ETS. Springer Nature, Gewerbestrasse, Switzerland."},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1177\/014662168801200305"},{"key":"e_1_3_2_14_2","volume-title":"Confirmatory Factor Analysis for Applied Research","author":"Brown Timothy A.","year":"2015","unstructured":"Timothy A. Brown. 2015. Confirmatory Factor Analysis for Applied Research (2nd. ed.). The Guilford Press, New York, NY.","edition":"2"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.clinthera.2014.04.006"},{"key":"e_1_3_2_16_2","volume-title":"TIC Educa\u00e7\u00e3o","author":"CGI.","year":"2019","unstructured":"CGI. 2019. TIC Educa\u00e7\u00e3o. Cetic, S\u00e3o Paulo, SP, Brazil."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.18637\/jss.v048.i06"},{"key":"e_1_3_2_18_2","first-page":"41","volume-title":"Proceedings of XXXI Brazilian Symposium on Informatics in Education","author":"Cruz Alves Nathalia Da","year":"2020","unstructured":"Nathalia Da Cruz Alves, Christiane Gresse Von Wangenheim, Matheus Alberto, and L\u00facia H. Martins-Pacheco. 2020. Uma proposta de avalia\u00e7\u00e3o da originalidade do produto no ensino de algoritmos e programa\u00e7\u00e3o na educa\u00e7\u00e3o b\u00e1sica. In Proceedings of XXXI Brazilian Symposium on Informatics in Education. Sociedade Brasileira de Computa\u00e7\u00e3o, Porto Alegre, Brazil, 41\u201350."},{"key":"e_1_3_2_19_2","volume-title":"The Theory and Practice of Item Response Theory","author":"De Ayala Rafael J.","year":"2022","unstructured":"Rafael J. De Ayala. 2022. The Theory and Practice of Item Response Theory (2nd. ed.). Guilford Press, New York, NY.","edition":"2"},{"key":"e_1_3_2_20_2","volume-title":"Scale Development: Theory and Applications","author":"DeVellis Robert F.","year":"2017","unstructured":"Robert F. DeVellis. 2017. Scale Development: Theory and Applications (4th. ed.). Sage, Los Angeles, CA.","edition":"4"},{"key":"e_1_3_2_21_2","unstructured":"Google. 2023. Google Teachable Machine. Retrieved September 2 2022 from https:\/\/teachablemachine.withgoogle.com\/"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10639-021-10570-8"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.3102\/003465430298487"},{"key":"e_1_3_2_24_2","first-page":"1","volume-title":"Proceedings of the Conference on Human Factors in Computing Systems","author":"Hitron Tom","year":"2019","unstructured":"Tom Hitron, Yoav Orlev, Iddo Wald, Ariel Shamir, Hadas Erel, and Oren Zuckerman. 2019. Can children understand machine learning concepts? The effect of uncovering black boxes. In Proceedings of the Conference on Human Factors in Computing Systems, 1\u201311."},{"key":"e_1_3_2_25_2","first-page":"157","volume-title":"Proceedings of the International Conference on Computational Thinking","author":"Ho Joshua W. K.","year":"2019","unstructured":"Joshua W. K. Ho and Matthew Scadding. 2019. Classroom activities for teaching artificial intelligence to primary school students. In Proceedings of the International Conference on Computational Thinking, 157\u2013159."},{"key":"e_1_3_2_26_2","volume-title":"AI in the UK: Ready, Willing and Able","author":"House of Lords","year":"2018","unstructured":"House of Lords. 2018. AI in the UK: Ready, Willing and Able. HL Paper 100, London, UK."},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.19173\/irrodl.v22i4.5474"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.3389\/feduc.2019.00045"},{"key":"e_1_3_2_29_2","volume-title":"Encyclopedia of Biostatistics","author":"Jackson Edward J.","year":"2005","unstructured":"Edward J. Jackson. 2005. Oblimin rotation. In Encyclopedia of Biostatistics. P. Armitage and T. Colton (Eds.), Vol. 6, John Wiley & Sons, Hoboken, NJ."},{"key":"e_1_3_2_30_2","first-page":"1","volume-title":"Proceedings of 2016 IEEE Frontiers in Education Conference (FIE)","author":"Kandlhofer Martin","year":"2016","unstructured":"Martin Kandlhofer, Gerald Steinbauer, Sabine Hirschmugl-Gaisch, and Petra Huber. 2016. Artificial intelligence and computer science in education: From kindergarten to university. In Proceedings of 2016 IEEE Frontiers in Education Conference (FIE), 1\u20139."},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2022.104562"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2021.106836"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2018.08.026"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1038\/nature14539"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/1929887.1929902"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376727"},{"key":"e_1_3_2_37_2","first-page":"227","volume-title":"Proceedings of the International Conference on Agile Software Development","author":"Lwakatare Lucy Ellen","year":"2019","unstructured":"Lucy Ellen Lwakatare, Aiswarya Raj, Jan Bosch, Helena Holmstr\u00f6m Olsson, and Ivica Crnkovic. 2019. A taxonomy of software engineering challenges for machine learning systems: An empirical investigation. In Proceedings of the International Conference on Agile Software Development. Springer, Montr\u00e9al, Canada, 227\u2013243."},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2014.09.012"},{"key":"e_1_3_2_39_2","unstructured":"MEC. 2018. Base Nacional Comum Curricular. Ministry of Education. Brazil. Retrieved January 6 2022 from http:\/\/basenacionalcomum.mec.gov.br\/"},{"key":"e_1_3_2_40_2","unstructured":"MEC. 2020. Census of Basic Education 2020. Ministry of Education. Brazil. Retrieved January 6 2022 from https:\/\/download.inep.gov.br\/publicacoes\/institucionais\/estatisticas_e_indicadores\/notas_estatisticas_censo_escolar_2020.pdf"},{"issue":"1","key":"e_1_3_2_41_2","first-page":"29","article-title":"A brief introduction to evidence-centered design","volume":"2003","author":"Mislevy Robert J.","year":"2003","unstructured":"Robert J. Mislevy, Russell G. Almond, and Janice F. Lukas. 2003. A brief introduction to evidence-centered design. ETS Research Report Series 2003, 1, (2003), i\u201329.","journal-title":"ETS Research Report Series"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.5555\/541177"},{"key":"e_1_3_2_43_2","volume-title":"Using R for Item Response Theory Model Applications","author":"Paek Insu","year":"2020","unstructured":"Insu Paek and Ki Cole. 2020. Using R for Item Response Theory Model Applications. Routledge, New York, NY."},{"key":"e_1_3_2_44_2","unstructured":"R Core Team. 2022. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing Vienna Austria. Retrieved from https:\/\/www.R-project.org\/"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.publhealth.25.102802.124410"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1080\/07370024.2020.1734931"},{"key":"e_1_3_2_47_2","unstructured":"Wiliam Revelle. 2022. psych: Procedures for Personality and Psychological Research. Northwestern University Evanston. Retrieved from https:\/\/CRAN.R-project.org\/package=psych"},{"key":"e_1_3_2_48_2","doi-asserted-by":"crossref","unstructured":"Steven P. Reise and Dennis A. Revicki. 2015. Handbook of Item Response Theory Modeling: Applications to Typical Performance Assessment. Routledge New York NY.","DOI":"10.4324\/9781315736013"},{"issue":"63","key":"e_1_3_2_49_2","first-page":"1","article-title":"LearningML: A tool to foster computational thinking skills through practical artificial intelligence projects","volume":"20","author":"Rodr\u00edguez-Garc\u00eda Juan D.","year":"2020","unstructured":"Juan D. Rodr\u00edguez-Garc\u00eda, Jes\u00fas Moreno-Le\u00f3n, Marcos Rom\u00e1n-Gonz\u00e1lez, and Gregorio Robles. 2020. LearningML: A tool to foster computational thinking skills through practical artificial intelligence projects. Distance Education Journal 20, 63, Article 7 (2020), 1\u201337.","journal-title":"Distance Education Journal"},{"key":"e_1_3_2_50_2","unstructured":"Royal Society. 2017. Machine Learning: The Power and Promise of Computers That Learn by Example. Retrieved January 6 2022 from royalsociety.org\/machine-learning"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.4324\/9781315637686"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF00117714"},{"key":"e_1_3_2_53_2","first-page":"1","volume-title":"Proceedings of Frontiers in Education Conference","author":"Santos Priscila S. C.","year":"2018","unstructured":"Priscila S. C. Santos, Luis G. J. Araujo, and Roberto A. Bittencourt. 2018. A mapping study of computational thinking and programming in Brazilian K-12 education. In Proceedings of Frontiers in Education Conference. IEEE, San Jose, CA, 1\u20138."},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF03372160"},{"key":"e_1_3_2_55_2","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1007\/978-1-4757-2691-6_5","volume-title":"Handbook of Modern Item Response Theory","author":"Samejima Fumiko","year":"1997","unstructured":"Fumiko Samejima. 1997. Graded response model. In Handbook of Modern Item Response Theory. Springer, New York, NY, 85\u2013100."},{"key":"e_1_3_2_56_2","volume-title":"Design Patterns for Assessing Internal Knowledge Representations","author":"Seeratan Kavita L.","year":"2008","unstructured":"Kavita L. Seeratan and Robert J. Mislevy. 2008. Design Patterns for Assessing Internal Knowledge Representations. PADI Technical Report 22. SRI International, Menlo Park."},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1007\/s13218-021-00729-3"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3328778.3366868"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.caeo.2023.100122"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2019.103798"},{"key":"e_1_3_2_61_2","volume-title":"Proceedings of the 2019 IJCAI EduAI Workshop","author":"Tang Danny","year":"2019","unstructured":"Danny Tang, Yuria Utsumi, and Natalie Lao. 2019. PIC: A personal image classification webtool for high school students. In Proceedings of the 2019 IJCAI EduAI Workshop. IJCAI, Macao, China."},{"key":"e_1_3_2_62_2","first-page":"9795","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Touretzky David","year":"2019","unstructured":"David Touretzky, Christina Gardner-McCune, Fred Martin, and Deborah Seehorn. 2019. Envisioning AI for K-12: What should every child know about AI? In Proceedings of the AAAI Conference on Artificial Intelligence. Honolulu, HA, 9795\u20139799."},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1007\/s40593-022-00314-1"},{"key":"e_1_3_2_64_2","volume-title":"The Research Methods Knowledge Base","author":"Trochim William M. K.","year":"2008","unstructured":"William M. K. Trochim and James P. Donnelly. 2008. The Research Methods Knowledge Base (3rd. ed.). Atomic Dog\/Cengage Learning, Mason, OH.","edition":"3"},{"key":"e_1_3_2_65_2","unstructured":"UNESCO. 2022. K-12 AI Curricula: A Mapping of Government-Endorsed AI Curricula. Retrieved June 6 2022 from https:\/\/unesdoc.unesco.org\/ark:\/48223\/pf0000380602"},{"key":"e_1_3_2_66_2","unstructured":"United Nations. 2015. The 17 Goals. Department of Economic and Social Affairs Sustainable Development. Retrieved June 6 2022 from https:\/\/sdgs.un.org\/goals"},{"key":"e_1_3_2_67_2","first-page":"26550","volume-title":"Proceedings of the 2016 ASEE Annual Conference & Exposition Proceedings. ASEE Conferences","author":"Yasar Osman","year":"2016","unstructured":"Osman Yasar, Peter Veronesi, Jose Maliekal, Leigh Little, Sounthone Vattana, and Ibrahim Yeter. 2016. Computational pedagogy: Fostering a new method of teaching. In Proceedings of the 2016 ASEE Annual Conference & Exposition Proceedings. ASEE Conferences, New Orleans, 26550."},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1007\/s40593-022-00325-y"},{"issue":"2","key":"e_1_3_2_69_2","first-page":"191","article-title":"Interpreting scales through scale anchoring","volume":"17","author":"Beaton Albert","year":"1992","unstructured":"Beaton Albert and Allen Nancy. 1992. Interpreting scales through scale anchoring. Journal of Educational Statistics 17, 2 (1992), 191\u2013204.","journal-title":"Journal of Educational Statistics"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177730491"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/3328778.3366896"},{"key":"e_1_3_2_72_2","first-page":"4","article-title":"Student assessment via graded response model","volume":"66","author":"Matteucci M.","year":"2006","unstructured":"Matteucci M. and Stracqualursi L. 2006. Student assessment via graded response model. Statistica 66 (2006), 4.","journal-title":"Statistica"},{"issue":"1","key":"e_1_3_2_73_2","first-page":"10","article-title":"Scoring rubric development: Validity and reliability","volume":"7","author":"Moskal B. M","year":"2000","unstructured":"Moskal B. M and Leydens J. A. 2000. Scoring rubric development: Validity and reliability. Practical Assessment, Research, and Evaluation 7, 1 (2000), 10.","journal-title":"Practical Assessment, Research, and Evaluation"},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2008.06.007"},{"issue":"9","key":"e_1_3_2_75_2","first-page":"27","article-title":"Classical test theory: An introduction to linear modeling approach to test and item analysis","volume":"2","author":"Bichi A. A.","year":"2016","unstructured":"Bichi A. A. 2016. Classical test theory: An introduction to linear modeling approach to test and item analysis. International Journal for Social Studies 2, 9 (2016), 27\u201333.","journal-title":"International Journal for Social Studies"},{"issue":"1","key":"e_1_3_2_76_2","first-page":"3","article-title":"Scoring rubrics: What, when and how","volume":"7","author":"Moskal B. M.","year":"2000","unstructured":"Moskal B. M. 2000. Scoring rubrics: What, when and how? Practical Assessment, Research, and Evaluation 7, 1 (2000), 3.","journal-title":"Practical Assessment, Research, and Evaluation"},{"key":"e_1_3_2_77_2","volume-title":"Classroom Assessment: Principles and Practice That Enhance Student Learning and Motivation","author":"McMillan James H.","year":"2018","unstructured":"James H. McMillan. 2018. Classroom Assessment: Principles and Practice That Enhance Student Learning and Motivation (7th. ed.). Pearson Education, Boston."},{"key":"e_1_3_2_78_2","volume-title":"Designing Effective Instruction","author":"Morrison G. R.","year":"2019","unstructured":"G. R. Morrison, S. M. Ross, J. R. Morrison, and H. K. Kalman. 2019. Designing Effective Instruction (8th. ed.). Wiley, Hoboken, NJ.","edition":"8"},{"key":"e_1_3_2_79_2","volume-title":"Standards for Educational and Psychological Testing","author":"AERA, APA","year":"2014","unstructured":"AERA, APA, and NCME. 2014. Standards for Educational and Psychological Testing. American Educational Research Association, Washington, DC."},{"issue":"4","key":"e_1_3_2_80_2","first-page":"1343","article-title":"National and international assessment in Brazil: The link between PISA and IDEB","volume":"43","author":"Villani M.","year":"2018","unstructured":"Villani M. and Oliveira D. A. 2018. National and international assessment in Brazil: The link between PISA and IDEB. Educa\u00e7\u00e3o & Realidade 43, 4 (2018), 1343\u20131362.","journal-title":"Educa\u00e7\u00e3o & Realidade"},{"key":"e_1_3_2_81_2","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-65368-6","volume-title":"Assessment and Teaching of 21st Century Skills: Research and Applications","author":"Care E.","year":"2018","unstructured":"Care E., Griffin P., and Wilson M. (Eds.). 2018. Assessment and Teaching of 21st Century Skills: Research and Applications. Springer International Publishing, Gewerbestrasse, Switzerland."},{"issue":"1","key":"e_1_3_2_82_2","first-page":"2","article-title":"Design and discovery in educational assessment: Evidence-centered design, psychometrics, and educational data mining","volume":"4","author":"Mislevy R. J.","year":"2012","unstructured":"Mislevy R. J. 2012. Design and discovery in educational assessment: Evidence-centered design, psychometrics, and educational data mining. Journal of Educational Data Mining 4, 1 (2012), 2.","journal-title":"Journal of Educational Data Mining"},{"key":"e_1_3_2_83_2","volume-title":"Evidence-Centered Assessment Design: Layers, Structures, and Terminology","author":"Mislevy R. J.","year":"2005","unstructured":"R. J. Mislevy and M. M. Riconscente. 2005. Evidence-Centered Assessment Design: Layers, Structures, and Terminology. PADI Technical Report 9. SRI International, Menlo Park."},{"key":"e_1_3_2_84_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2008.06.007"}],"container-title":["ACM Transactions on Computing Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3749993","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,18]],"date-time":"2025-09-18T17:18:29Z","timestamp":1758215909000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3749993"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,18]]},"references-count":83,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,12,31]]}},"alternative-id":["10.1145\/3749993"],"URL":"https:\/\/doi.org\/10.1145\/3749993","relation":{},"ISSN":["1946-6226"],"issn-type":[{"type":"electronic","value":"1946-6226"}],"subject":[],"published":{"date-parts":[[2025,9,18]]},"assertion":[{"value":"2023-08-09","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-07-05","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-09-18","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}