{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T01:44:21Z","timestamp":1787017461638,"version":"3.56.0"},"reference-count":85,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2024,4,16]],"date-time":"2024-04-16T00:00:00Z","timestamp":1713225600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Educ."],"published-print":{"date-parts":[[2024,6,30]]},"abstract":"<jats:p>Machine learning (ML) models are fundamentally shaped by data, and building inclusive ML systems requires significant considerations around how to design representative datasets. Yet, few novice-oriented ML modeling tools are designed to foster hands-on learning of dataset design practices, including how to design for data diversity and inspect for data quality.<\/jats:p>\n                  <jats:p>To this end, we outline a set of four data design practices (DDPs) for designing inclusive ML models and share how we designed a tablet-based application called Co-ML to foster learning of DDPs through a collaborative ML model building experience. With Co-ML, beginners can build image classifiers through a distributed experience where data is synchronized across multiple devices, enabling multiple users to iteratively refine ML datasets in discussion and coordination with their peers.<\/jats:p>\n                  <jats:p>We deployed Co-ML in a 2-week-long educational AIML Summer Camp, where youth ages 13\u201318 worked in groups to build custom ML-powered mobile applications. Our analysis reveals how multi-user model building with Co-ML, in the context of student-driven projects created during the summer camp, supported development of DDPs including incorporating data diversity, evaluating model performance, and inspecting for data quality. Additionally, we found that students\u2019 attempts to improve model performance often prioritized learnability over class balance. Through this work, we highlight how the combination of collaboration, model testing interfaces, and student-driven projects can empower learners to actively engage in exploring the role of data in ML systems.<\/jats:p>","DOI":"10.1145\/3641552","type":"journal-article","created":{"date-parts":[[2024,1,22]],"date-time":"2024-01-22T06:51:30Z","timestamp":1705906290000},"page":"1-37","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":20,"title":["Co-ML: Collaborative Machine Learning Model Building for Developing Dataset Design Practices"],"prefix":"10.1145","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4972-4061","authenticated-orcid":false,"given":"Tiffany","family":"Tseng","sequence":"first","affiliation":[{"name":"Apple, Seattle, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6848-2876","authenticated-orcid":false,"given":"Matt J.","family":"Davidson","sequence":"additional","affiliation":[{"name":"Apple, Austin, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8777-2374","authenticated-orcid":false,"given":"Luis","family":"Morales-Navarro","sequence":"additional","affiliation":[{"name":"University of Pennsylvania, Philadelphia, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7710-8023","authenticated-orcid":false,"given":"Jennifer King","family":"Chen","sequence":"additional","affiliation":[{"name":"Apple, Seattle, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1907-0303","authenticated-orcid":false,"given":"Victoria","family":"Delaney","sequence":"additional","affiliation":[{"name":"Stanford University, Stanford, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-0134-9836","authenticated-orcid":false,"given":"Mark","family":"Leibowitz","sequence":"additional","affiliation":[{"name":"Apple, Seattle, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-1499-8869","authenticated-orcid":false,"given":"Jazbo","family":"Beason","sequence":"additional","affiliation":[{"name":"Apple, Seattle, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1359-7120","authenticated-orcid":false,"given":"R. Benjamin","family":"Shapiro","sequence":"additional","affiliation":[{"name":"Apple, Seattle, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,4,16]]},"reference":[{"key":"e_1_3_3_2_2","first-page":"1","volume-title":"Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems","author":"Agassi Adam","year":"2019","unstructured":"Adam Agassi, Hadas Erel, Iddo Yehoshua Wald, and Oren Zuckerman. 2019. Scratch nodes ML: A playful system for children to create gesture recognition classifiers. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 1\u20136."},{"key":"e_1_3_3_3_2","first-page":"1","volume-title":"International Workshop on Education in Artificial Intelligence k-12 (Eduai\u201919)","author":"Ali Safinah","year":"2019","unstructured":"Safinah Ali, Blakeley H Payne, Randi Williams, Hae Won Park, and Cynthia Breazeal. 2019. Constructionism, ethics, and creativity: Developing primary and middle school artificial intelligence education. In International Workshop on Education in Artificial Intelligence k-12 (Eduai\u201919), Vol. 2. 1\u20134."},{"key":"e_1_3_3_4_2","unstructured":"Apple. 2022. CreateML. Retrieved from https:\/\/developer.apple.com\/machine-learning\/create-ml\/"},{"issue":"4","key":"e_1_3_3_5_2","first-page":"126","article-title":"Designing with and for youth: A participatory design research approach for critical machine learning education","volume":"25","author":"Irgens Golnaz Arastoopour","year":"2022","unstructured":"Golnaz Arastoopour Irgens, Ibrahim Adisa, Cinamon Bailey, and Hazel Vega Quesada. 2022. Designing with and for youth: A participatory design research approach for critical machine learning education. Educational Technology & Society 25, 4 (2022), 126\u2013141.","journal-title":"Educational Technology & Society"},{"key":"e_1_3_3_6_2","doi-asserted-by":"crossref","first-page":"100541","DOI":"10.1016\/j.ijcci.2022.100541","article-title":"Characterizing children\u2019s conceptual knowledge and computational practices in a critical machine learning educational program","volume":"34","author":"Irgens Golnaz Arastoopour","year":"2022","unstructured":"Golnaz Arastoopour Irgens, Hazel Vega, Ibrahim Adisa, and Cinamon Bailey. 2022. Characterizing children\u2019s conceptual knowledge and computational practices in a critical machine learning educational program. International Journal of Child-Computer Interaction 34 (2022), 100541.","journal-title":"International Journal of Child-Computer Interaction"},{"key":"e_1_3_3_7_2","volume-title":"Passages from the Life of a Philosopher","author":"Babbage Charles","year":"2022","unstructured":"Charles Babbage. 2022. Passages from the Life of a Philosopher. DigiCat."},{"issue":"3","key":"e_1_3_3_8_2","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1207\/S15327809JLS1203_1","article-title":"When smart groups fail","volume":"12","author":"Barron Brigid","year":"2003","unstructured":"Brigid Barron. 2003. When smart groups fail. The Journal of the Learning Sciences 12, 3 (2003), 307\u2013359.","journal-title":"The Journal of the Learning Sciences"},{"key":"e_1_3_3_9_2","doi-asserted-by":"crossref","first-page":"900","DOI":"10.1145\/3408877.3432517","volume-title":"Proceedings of the 52nd ACM Technical Symposium on Computer Science Education","author":"Bigman Maxwell","year":"2021","unstructured":"Maxwell Bigman, Ethan Roy, Jorge Garcia, Miroslav Suzara, Kaili Wang, and Chris Piech. 2021. PearProgram: A more fruitful approach to pair programming. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education. 900\u2013906."},{"issue":"8","key":"e_1_3_3_10_2","doi-asserted-by":"crossref","first-page":"37","DOI":"10.2307\/1176492","article-title":"Learning with peers: From small group cooperation to collaborative communities","volume":"25","author":"Blumenfeld Phyllis C.","year":"1996","unstructured":"Phyllis C. Blumenfeld, Ronald W. Marx, Elliot Soloway, and Joseph Krajcik. 1996. Learning with peers: From small group cooperation to collaborative communities. Educational Researcher 25, 8 (1996), 37\u201339.","journal-title":"Educational Researcher"},{"key":"e_1_3_3_11_2","volume-title":"How People Learn","unstructured":"John D. Bransford, Ann L. Brown, and Rodney R. Cocking. 2000. How People Learn. Vol. 11, National Academy Press, Washington, DC."},{"issue":"1","key":"e_1_3_3_12_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1921607.1921609","article-title":"The case for pair programming in the computer science classroom","volume":"11","author":"Braught Grant","year":"2011","unstructured":"Grant Braught, Tim Wahls, and L Marlin Eby. 2011. The case for pair programming in the computer science classroom. ACM Transactions on Computing Education (TOCE) 11, 1 (2011), 1\u201321.","journal-title":"ACM Transactions on Computing Education (TOCE)"},{"key":"e_1_3_3_13_2","first-page":"77","volume-title":"Conference on Fairness, Accountability and Transparency","author":"Buolamwini Joy","year":"2018","unstructured":"Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on Fairness, Accountability and Transparency. PMLR, 77\u201391."},{"key":"e_1_3_3_14_2","first-page":"1","volume-title":"Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems","author":"Carney Michelle","year":"2020","unstructured":"Michelle Carney, Barron Webster, Irene Alvarado, Kyle Phillips, Noura Howell, Jordan Griffith, Jonas Jongejan, Amit Pitaru, and Alexander Chen. 2020. Teachable machine: Approachable web-based tool for exploring machine learning classification. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. 1\u20138."},{"issue":"1","key":"e_1_3_3_15_2","first-page":"509","article-title":"Grounded theory: Objectivist and constructivist methods","volume":"2","author":"Charmaz Kathy","year":"2000","unstructured":"Kathy Charmaz. 2000. Grounded theory: Objectivist and constructivist methods. Handbook of Qualitative Research 2, 1 (2000), 509\u2013535.","journal-title":"Handbook of Qualitative Research"},{"key":"e_1_3_3_16_2","volume-title":"Statistical Power Analysis for the Behavioral Sciences","author":"Cohen Jacob","year":"1988","unstructured":"Jacob Cohen. 1988. Statistical Power Analysis for the Behavioral Sciences. Routledge."},{"key":"e_1_3_3_17_2","first-page":"296","volume-title":"Ethics of Data and Analytics","author":"Dastin Jeffrey","year":"2018","unstructured":"Jeffrey Dastin. 2018. Amazon scraps secret AI recruiting tool that showed bias against women. In Ethics of Data and Analytics. Auerbach Publications, 296\u2013299."},{"key":"e_1_3_3_18_2","unstructured":"Elise Deitrick Brian O\u2019Connell and R. Benjamin Shapiro. 2014. The discourse of creative problem solving in childhood engineering education. Boulder CO: International Society of the Learning Sciences."},{"key":"e_1_3_3_19_2","unstructured":"Elise Deitrick R. Benjamin Shapiro and Brian Gravel. 2016. How do we assess equity in programming pairs?Singapore: International Society of the Learning Sciences."},{"key":"e_1_3_3_20_2","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1145\/143457.143468","volume-title":"Proceedings of the 1992 ACM Conference on Computer-Supported Cooperative Work","author":"Dourish Paul","year":"1992","unstructured":"Paul Dourish and Victoria Bellotti. 1992. Awareness and coordination in shared workspaces. In Proceedings of the 1992 ACM Conference on Computer-Supported Cooperative Work. 107\u2013114."},{"key":"e_1_3_3_21_2","volume-title":"Growing up with AI: Cognimates: from Coding to Teaching Machines","author":"Druga Stefania","year":"2018","unstructured":"Stefania Druga. 2018. Growing up with AI: Cognimates: from Coding to Teaching Machines. Ph.D. Dissertation. Massachusetts Institute of Technology."},{"key":"e_1_3_3_22_2","first-page":"1","volume-title":"Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems","author":"Druga Stefania","year":"2022","unstructured":"Stefania Druga, Fee Lia Christoph, and Amy J. Ko. 2022. Family as a third space for AI literacies: How do children and parents learn about AI together? In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1\u201317."},{"key":"e_1_3_3_23_2","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1145\/3502718.3524782","volume-title":"Proceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 1","author":"Druga Stefania","year":"2022","unstructured":"Stefania Druga, Nancy Otero, and Amy J. Ko. 2022. The landscape of teaching resources for ai education. In Proceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 1. 96\u2013102."},{"key":"e_1_3_3_24_2","first-page":"1","volume-title":"2021 IEEE Symposium on Visual Languages and Human-Centric Computing (VL\/HCC)","author":"Dwivedi Utkarsh","year":"2021","unstructured":"Utkarsh Dwivedi, Jaina Gandhi, Raj Parikh, Merijke Coenraad, Elizabeth Bonsignore, and Hernisa Kacorri. 2021. Exploring machine teaching with children. In 2021 IEEE Symposium on Visual Languages and Human-Centric Computing (VL\/HCC). IEEE, 1\u201311."},{"key":"e_1_3_3_25_2","doi-asserted-by":"crossref","first-page":"179027","DOI":"10.1109\/ACCESS.2019.2956136","article-title":"Gentle introduction to artificial intelligence for high-school students using scratch","volume":"7","author":"Estevez Julian","year":"2019","unstructured":"Julian Estevez, Gorka Garate, and Manuel Gra\u00f1a. 2019. Gentle introduction to artificial intelligence for high-school students using scratch. IEEE Access 7 (2019), 179027\u2013179036.","journal-title":"IEEE Access"},{"key":"e_1_3_3_26_2","volume-title":"Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor","author":"Eubanks Virginia","year":"2018","unstructured":"Virginia Eubanks. 2018. Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor. St. Martin\u2019s Press."},{"issue":"4","key":"e_1_3_3_27_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3294008","article-title":"Machine learning education for artists, musicians, and other creative practitioners","volume":"19","author":"Fiebrink Rebecca","year":"2019","unstructured":"Rebecca Fiebrink. 2019. Machine learning education for artists, musicians, and other creative practitioners. ACM Transactions on Computing Education (TOCE) 19, 4 (2019), 1\u201332.","journal-title":"ACM Transactions on Computing Education (TOCE)"},{"key":"e_1_3_3_28_2","unstructured":"Center for Democracy and Technology. 2019. AI & Machine Learning. Retrieved from https:\/\/cdt.org\/ai-machine-learning\/"},{"key":"e_1_3_3_29_2","doi-asserted-by":"crossref","DOI":"10.4324\/9780203793206","volume-title":"Discovery of Grounded Theory: Strategies for Qualitative Research","author":"Glaser Barney G.","year":"2017","unstructured":"Barney G. Glaser and Anselm L. Strauss. 2017. Discovery of Grounded Theory: Strategies for Qualitative Research. Routledge."},{"key":"e_1_3_3_30_2","unstructured":"Google. 2023. Colab. Retrieved from https:\/\/colab.research.google.com"},{"key":"e_1_3_3_31_2","unstructured":"MIT Media Lab Personal Robots Group and MIT STEP Lab. 2023. DAILy Curriculum for Middle School Students. Retrieved from https:\/\/raise.mit.edu\/daily\/index.html"},{"key":"e_1_3_3_32_2","doi-asserted-by":"crossref","first-page":"563","DOI":"10.1145\/3202185.3210776","volume-title":"Proceedings of the 17th ACM Conference on Interaction Design and Children","author":"Hitron Tom","year":"2018","unstructured":"Tom Hitron, Iddo Wald, Hadas Erel, and Oren Zuckerman. 2018. Introducing children to machine learning concepts through hands-on experience. In Proceedings of the 17th ACM Conference on Interaction Design and Children. 563\u2013568."},{"key":"e_1_3_3_33_2","doi-asserted-by":"crossref","unstructured":"Cindy Hmelo-Silver Heisawn Jeong Roosevelt Faulkner and Kylie Hartley. 2017. Computer-supported collaborative learning in STEM domains: Towards a meta-synthesis. (2017).","DOI":"10.24251\/HICSS.2017.251"},{"key":"e_1_3_3_34_2","volume-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems","author":"Hohman Fred","year":"2020","unstructured":"Fred Hohman, Kanit Wongsuphasawat, Mary Beth Kery, and Kayur Patel. 2020. Understanding and visualizing data iteration in machine learning. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM. DOI:10.1145\/3313831.3376177"},{"key":"e_1_3_3_35_2","unstructured":"Aspen Hopkins Fred Hohman Luca Zappella Xavier Suau Cuadros and Dominik Moritz. 2023. Designing data: Proactive data collection and iteration for machine learning. arXiv:2301.10319. Retrieved from https:\/\/arxiv.org\/abs\/cs\/2301.10319"},{"key":"e_1_3_3_36_2","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1109\/VLHCC.2013.6645252","volume-title":"2013 IEEE Symposium on Visual Languages and Human Centric Computing","author":"Jones Danielle L.","year":"2013","unstructured":"Danielle L. Jones and Scott D. Fleming. 2013. What use is a backseat driver? A qualitative investigation of pair programming. In 2013 IEEE Symposium on Visual Languages and Human Centric Computing. IEEE, 103\u2013110."},{"key":"e_1_3_3_37_2","first-page":"15551","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","volume":"35","author":"Jordan Brian","year":"2021","unstructured":"Brian Jordan, Nisha Devasia, Jenna Hong, Randi Williams, and Cynthia Breazeal. 2021. PoseBlocks: A toolkit for creating (and dancing) with AI. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 15551\u201315559."},{"issue":"3","key":"e_1_3_3_38_2","doi-asserted-by":"crossref","first-page":"1130","DOI":"10.1111\/bjet.13088","article-title":"Constructionism and AI: A history and possible futures","volume":"52","author":"Kahn Ken","year":"2021","unstructured":"Ken Kahn and Niall Winters. 2021. Constructionism and AI: A history and possible futures. British Journal of Educational Technology 52, 3 (2021), 1130\u20131142.","journal-title":"British Journal of Educational Technology"},{"key":"e_1_3_3_39_2","first-page":"1","volume-title":"FabLearn Europe\/MakeEd 2021-An International Conference on Computing, Design and Making in Education","author":"Kaspersen Magnus Hoeholt","year":"2021","unstructured":"Magnus Hoeholt Kaspersen, Karl-Emil Kjaer Bilstrup, Maarten Van Mechelen, Arthur Hjorth, Niels Olof Bouvin, and Marianne Graves Petersen. 2021. VotestratesML: A high school learning tool for exploring machine learning and its societal implications. In FabLearn Europe\/MakeEd 2021-An International Conference on Computing, Design and Making in Education. 1\u201310."},{"key":"e_1_3_3_40_2","first-page":"1","volume-title":"Proceedings of the Australasian Computer Science Week Multiconference","author":"Khan Nazish Zaman","year":"2016","unstructured":"Nazish Zaman Khan and Andrew Luxton-Reilly. 2016. Is computing for social good the solution to closing the gender gap in computer science? In Proceedings of the Australasian Computer Science Week Multiconference. 1\u20135."},{"key":"e_1_3_3_41_2","doi-asserted-by":"crossref","first-page":"102562","DOI":"10.1016\/j.ijhcs.2020.102562","article-title":"Talking datasets\u2013understanding data sensemaking behaviours","volume":"146","author":"Koesten Laura","year":"2021","unstructured":"Laura Koesten, Kathleen Gregory, Paul Groth, and Elena Simperl. 2021. Talking datasets\u2013understanding data sensemaking behaviours. International Journal of Human\u2013Computer Studies 146 (2021), 102562.","journal-title":"International Journal of Human\u2013Computer Studies"},{"key":"e_1_3_3_42_2","volume-title":"Machine Learning for Kids: A Project-Based Introduction to Artificial Intelligence","author":"Lane Dale","year":"2021","unstructured":"Dale Lane. 2021. Machine Learning for Kids: A Project-Based Introduction to Artificial Intelligence. No Starch Press."},{"issue":"3","key":"e_1_3_3_43_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3484495","article-title":"In the black mirror: Youth investigations into artificial intelligence","volume":"22","author":"Lee Clifford H.","year":"2022","unstructured":"Clifford H. Lee, Nimah Gobir, Alex Gurn, and Elisabeth Soep. 2022. In the black mirror: Youth investigations into artificial intelligence. ACM Transactions on Computing Education 22, 3 (2022), 1\u201325.","journal-title":"ACM Transactions on Computing Education"},{"key":"e_1_3_3_44_2","first-page":"191","volume-title":"Proceedings of the 52nd ACM Technical Symposium on Computer Science Education","author":"Lee Irene","year":"2021","unstructured":"Irene Lee, Safinah Ali, Helen Zhang, Daniella DiPaola, and Cynthia Breazeal. 2021. Developing middle school students\u2019 AI literacy. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education. 191\u2013197."},{"key":"e_1_3_3_45_2","first-page":"1","volume-title":"Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems","author":"Long Duri","year":"2020","unstructured":"Duri Long and Brian Magerko. 2020. What is AI literacy? Competencies and design considerations. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1\u201316."},{"key":"e_1_3_3_46_2","first-page":"1","volume-title":"Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems","author":"Long Duri","year":"2022","unstructured":"Duri Long, Anthony Teachey, and Brian Magerko. 2022. Family learning talk in AI literacy learning activities. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1\u201320."},{"key":"e_1_3_3_47_2","doi-asserted-by":"crossref","first-page":"832","DOI":"10.1145\/3328778.3366943","volume-title":"Proceedings of the 51st ACM Technical Symposium on Computer Science Education","author":"Lytle Nicholas","year":"2020","unstructured":"Nicholas Lytle, Alexandra Milliken, Veronica Catet\u00e9, and Tiffany Barnes. 2020. Investigating different assignment designs to promote collaboration in block-based environments. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education. 832\u2013838."},{"key":"e_1_3_3_48_2","volume-title":"Cambridge Handbook of Computing Education Research","author":"Margulieux Lauren E.","year":"2019","unstructured":"Lauren E. Margulieux, Brian Dorn, and Kristin A. Searle. 2019. Learning sciences for computing education. In Cambridge Handbook of Computing Education Research. Sally A. Fincher and Anthony V. Robins (Eds.). Cambridge: Cambridge University Press."},{"issue":"2","key":"e_1_3_3_49_2","doi-asserted-by":"crossref","first-page":"283","DOI":"10.15388\/infedu.2020.14","article-title":"Teaching machine learning in school: A systematic mapping of the state of the art","volume":"19","author":"Marques L\u00edvia S.","year":"2020","unstructured":"L\u00edvia S. Marques, Christiane Gresse von Wangenheim, and Jean CR Hauck. 2020. Teaching machine learning in school: A systematic mapping of the state of the art. Informatics in Education 19, 2 (2020), 283\u2013321.","journal-title":"Informatics in Education"},{"key":"e_1_3_3_50_2","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1145\/3375627.3375832","volume-title":"Proceedings of the AAAI\/ACM Conference on AI, Ethics, and Society","author":"Mitchell Margaret","year":"2020","unstructured":"Margaret Mitchell, Dylan Baker, Nyalleng Moorosi, Emily Denton, Ben Hutchinson, Alex Hanna, Timnit Gebru, and Jamie Morgenstern. 2020. Diversity and inclusion metrics in subset selection. In Proceedings of the AAAI\/ACM Conference on AI, Ethics, and Society. 117\u2013123."},{"key":"e_1_3_3_51_2","doi-asserted-by":"crossref","DOI":"10.18574\/nyu\/9781479833641.001.0001","volume-title":"Algorithms of Oppression","author":"Noble Safiya Umoja","year":"2018","unstructured":"Safiya Umoja Noble. 2018. Algorithms of Oppression. New York University Press."},{"key":"e_1_3_3_52_2","unstructured":"Department of Agriculture. 2022. Child Nutrition Programs: Income Eligibility Guidelines. Retrieved from https:\/\/www.govinfo.gov\/content\/pkg\/FR-2022-02-16\/pdf\/2022-03261.pdf"},{"key":"e_1_3_3_53_2","unstructured":"Blakeley H. Payne. 2019. An ethics of artificial intelligence curriculum for middle school students. MIT Media Lab Personal Robots Group. Accessed October 2019."},{"key":"e_1_3_3_54_2","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1145\/2839509.2844642","volume-title":"Proceedings of the 47th ACM Technical Symposium on Computing Science Education","author":"Porter Leo","year":"2016","unstructured":"Leo Porter, Dennis Bouvier, Quintin Cutts, Scott Grissom, Cynthia Lee, Robert McCartney, Daniel Zingaro, and Beth Simon. 2016. A multi-institutional study of peer instruction in introductory computing. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education. 358\u2013363."},{"issue":"5","key":"e_1_3_3_55_2","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1080\/07370024.2020.1734931","article-title":"Interactive machine teaching: A human-centered approach to building machine-learned models","volume":"35","author":"Ramos Gonzalo","year":"2020","unstructured":"Gonzalo Ramos, Christopher Meek, Patrice Simard, Jina Suh, and Soroush Ghorashi. 2020. Interactive machine teaching: A human-centered approach to building machine-learned models. Human\u2013Computer Interaction 35, 5-6 (2020), 413\u2013451.","journal-title":"Human\u2013Computer Interaction"},{"key":"e_1_3_3_56_2","unstructured":"Thomas C. Redman. 2018. If your data is bad your machine learning tools are useless. Harvard Business Review 2 (2018). Retrieved from https:\/\/hbr.org\/2018\/04\/if-your-data-is-bad-your-machine-learning-tools-are-useless"},{"key":"e_1_3_3_57_2","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1145\/3372782.3406252","volume-title":"Proceedings of the 2020 ACM Conference on International Computing Education Research","author":"Register Yim","year":"2020","unstructured":"Yim Register and Amy J. Ko. 2020. Learning machine learning with personal data helps stakeholders ground advocacy arguments in model mechanics. In Proceedings of the 2020 ACM Conference on International Computing Education Research. 67\u201378."},{"key":"e_1_3_3_58_2","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1145\/3017680.3017748","volume-title":"Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE \u201917)","author":"Rodr\u00edguez Fernando J.","year":"2017","unstructured":"Fernando J. Rodr\u00edguez, Kimberly Michelle Price, and Kristy Elizabeth Boyer. 2017. Exploring the pair programming process: Characteristics of effective collaboration. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (SIGCSE \u201917). Association for Computing Machinery, New York, NY, 507\u2013512. DOI:10.1145\/3017680.3017748"},{"issue":"3","key":"e_1_3_3_59_2","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1207\/s15327809jls0203_1","article-title":"Learning by collaborating: Convergent conceptual change","volume":"2","author":"Roschelle Jeremy","year":"1992","unstructured":"Jeremy Roschelle. 1992. Learning by collaborating: Convergent conceptual change. The Journal of the Learning Sciences 2, 3 (1992), 235\u2013276.","journal-title":"The Journal of the Learning Sciences"},{"key":"e_1_3_3_60_2","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1007\/978-3-642-85098-1_5","volume-title":"Computer Supported Collaborative Learning","author":"Roschelle Jeremy","year":"1995","unstructured":"Jeremy Roschelle and Stephanie D. Teasley. 1995. The construction of shared knowledge in collaborative problem solving. In Computer Supported Collaborative Learning. Springer, 69\u201397."},{"key":"e_1_3_3_61_2","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1109\/CSEET.2011.5876082","volume-title":"2011 24th IEEE-CS Conference on Software Engineering Education and Training (CSEE&T)","author":"Salleh Norsaremah","year":"2011","unstructured":"Norsaremah Salleh, Emilia Mendes, and John Grundy. 2011. The effects of openness to experience on pair programming in a higher education context. In 2011 24th IEEE-CS Conference on Software Engineering Education and Training (CSEE&T). IEEE, 149\u2013158."},{"key":"e_1_3_3_62_2","first-page":"1","volume-title":"Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems","author":"Sambasivan Nithya","year":"2021","unstructured":"Nithya Sambasivan, Shivani Kapania, Hannah Highfill, Diana Akrong, Praveen Paritosh, and Lora M Aroyo. 2021. \u201cEveryone wants to do the model work, not the data work\u201d: Data cascades in high-stakes AI. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1\u201315."},{"key":"e_1_3_3_63_2","doi-asserted-by":"crossref","first-page":"100072","DOI":"10.1016\/j.caeo.2021.100072","article-title":"Exploring teachers\u2019 preconceptions of teaching machine learning in high school: A preliminary insight from Africa","volume":"3","author":"Sanusi Ismaila Temitayo","year":"2022","unstructured":"Ismaila Temitayo Sanusi, Solomon Sunday Oyelere, and Joseph Olamide Omidiora. 2022. Exploring teachers\u2019 preconceptions of teaching machine learning in high school: A preliminary insight from Africa. Computers and Education Open 3 (2022), 100072.","journal-title":"Computers and Education Open"},{"key":"e_1_3_3_64_2","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1109\/VLHCC.2019.8818895","volume-title":"2019 IEEE Symposium on Visual Languages and Human-Centric Computing (VL\/HCC)","author":"Selwyn-Smith Ben","year":"2019","unstructured":"Ben Selwyn-Smith, Craig Anslow, Michael Homer, and James R. Wallace. 2019. Co-located collaborative block-based programming. In 2019 IEEE Symposium on Visual Languages and Human-Centric Computing (VL\/HCC). IEEE, 107\u2013116."},{"key":"e_1_3_3_65_2","doi-asserted-by":"crossref","unstructured":"R. Benjamin Shapiro and Rebecca Fiebrink. 2019. Introduction to the special section: Launching an agenda for research on learning machine learning. ACM Transactions on Computing Education (TOCE) 19 4 (2019) 1--6.","DOI":"10.1145\/3354136"},{"key":"e_1_3_3_66_2","unstructured":"Patrice Y. Simard Saleema Amershi David M. Chickering Alicia Edelman Pelton Soroush Ghorashi Christopher Meek Gonzalo Ramos Jina Suh Johan Verwey Mo Wang and John Wernsing. 2017. Machine teaching: A new paradigm for building machine learning systems. arXiv:1707.06742. Retrieved from https:\/\/arxiv.org\/abs\/cs\/1707.06742"},{"key":"e_1_3_3_67_2","volume-title":"Basics of Qualitative Research","author":"Strauss Anselm","year":"1990","unstructured":"Anselm Strauss and Juliet Corbin. 1990. Basics of Qualitative Research. Sage publications."},{"key":"e_1_3_3_68_2","volume-title":"Empowering Novices to Understand and Use Machine Learning With Personalized Image Classification Models, Intuitive Analysis Tools, and MIT App Inventor","author":"Tang Danny","year":"2019","unstructured":"Danny Tang. 2019. Empowering Novices to Understand and Use Machine Learning With Personalized Image Classification Models, Intuitive Analysis Tools, and MIT App Inventor. Ph.D. Dissertation. Massachusetts Institute of Technology."},{"issue":"2","key":"e_1_3_3_69_2","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1145\/269012.269021","article-title":"Examining data quality","volume":"41","author":"Tayi Giri Kumar","year":"1998","unstructured":"Giri Kumar Tayi and Donald P. Ballou. 1998. Examining data quality. Commun. ACM 41, 2 (1998), 54\u201357.","journal-title":"Commun. ACM"},{"key":"e_1_3_3_70_2","first-page":"1","volume-title":"Proceedings of the 21st Koli Calling International Conference on Computing Education Research","author":"Tedre Matti","year":"2021","unstructured":"Matti Tedre, Peter Denning, and Tapani Toivonen. 2021. CT 2.0. In Proceedings of the 21st Koli Calling International Conference on Computing Education Research. 1\u20138."},{"key":"e_1_3_3_71_2","doi-asserted-by":"crossref","unstructured":"David Touretzky Christina Gardner-McCune and Deborah Seehorn. 2022. Machine learning and the five big ideas in AI. International Journal of Artificial Intelligence in Education 33 2 (2022) 233--266.","DOI":"10.1007\/s40593-022-00314-1"},{"key":"e_1_3_3_72_2","unstructured":"David S. Touretzky Christina Gardner-McCune Fred Martin and Deborah Seehorn. 2019. K-12 Guidelines for Artificial Intelligence: What Students Should Know. Retrieved from https:\/\/github.com\/touretzkyds\/ai4k12\/raw\/master\/documents\/ISTE_2019_Presentation_website_final.pdf\/"},{"key":"e_1_3_3_73_2","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1145\/3459990.3460694","volume-title":"Interaction Design and Children","author":"Tseng Tiffany","year":"2021","unstructured":"Tiffany Tseng, Yumiko Murai, Natalie Freed, Deanna Gelosi, Tung D Ta, and Yoshihiro Kawahara. 2021. PlushPal: Storytelling with interactive plush toys and machine learning. In Interaction Design and Children. 236\u2013245."},{"key":"e_1_3_3_74_2","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1145\/2839509.2844620","volume-title":"Proceedings of the 47th ACM Technical Symposium on Computing Science Education","author":"Vachovsky Marie E.","year":"2016","unstructured":"Marie E. Vachovsky, Grace Wu, Sorathan Chaturapruek, Olga Russakovsky, Richard Sommer, and Li Fei-Fei. 2016. Toward more gender diversity in CS through an artificial intelligence summer program for high school girls. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education. 303\u2013308."},{"key":"e_1_3_3_75_2","first-page":"15655","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","volume":"35","author":"Brummelen Jessica Van","year":"2021","unstructured":"Jessica Van Brummelen, Tommy Heng, and Viktoriya Tabunshchyk. 2021. Teaching tech to talk: K-12 conversational artificial intelligence literacy curriculum and development tools. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 15655\u201315663."},{"key":"e_1_3_3_76_2","first-page":"1","volume-title":"2020 IEEE Frontiers in Education Conference (FIE)","author":"Vartiainen Henriikka","year":"2020","unstructured":"Henriikka Vartiainen, Tapani Toivonen, Ilkka Jormanainen, Juho Kahila, Matti Tedre, and Teemu Valtonen. 2020. Machine learning for middle-schoolers: Children as designers of machine-learning apps. In 2020 IEEE Frontiers in Education Conference (FIE). IEEE, 1\u20139."},{"key":"e_1_3_3_77_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3359141","article-title":"How data scientists use computational notebooks for real-time collaboration","volume":"3","author":"Wang April Yi","year":"2019","unstructured":"April Yi Wang, Anant Mittal, Christopher Brooks, and Steve Oney. 2019. How data scientists use computational notebooks for real-time collaboration. Proceedings of the ACM on Human\u2013Computer Interaction 3, CSCW (2019), 1\u201330.","journal-title":"Proceedings of the ACM on Human\u2013Computer Interaction"},{"issue":"1","key":"e_1_3_3_78_2","first-page":"1","article-title":"A survey of transfer learning","volume":"3","author":"Weiss Karl","year":"2016","unstructured":"Karl Weiss, Taghi M. Khoshgoftaar, and DingDing Wang. 2016. A survey of transfer learning. Journal of Big Data 3, 1 (2016), 1\u201340.","journal-title":"Journal of Big Data"},{"issue":"1","key":"e_1_3_3_79_2","doi-asserted-by":"crossref","first-page":"4\u2013es","DOI":"10.1145\/1060071.1060075","article-title":"Pair-programming helps female computer science students","volume":"4","author":"Werner Linda L.","year":"2004","unstructured":"Linda L. Werner, Brian Hanks, and Charlie McDowell. 2004. Pair-programming helps female computer science students. Journal on Educational Resources in Computing (JERIC) 4, 1 (2004), 4\u2013es.","journal-title":"Journal on Educational Resources in Computing (JERIC)"},{"key":"e_1_3_3_80_2","doi-asserted-by":"crossref","unstructured":"Randi Williams Safinah Ali Nisha Devasia Daniella DiPaola Jenna Hong Stephen P. Kaputsos Brian Jordan and Cynthia Breazeal. 2022. AI+ ethics curricula for middle school youth: Lessons learned from three project-based curricula. International Journal of Artificial Intelligence in Education 33 2 (2022) 325--383.","DOI":"10.1007\/s40593-022-00298-y"},{"key":"e_1_3_3_81_2","unstructured":"H. James Wilson and Paul R. Daugherty. 2020. Small Data Can Play a Big Role in AI. Harvard Business Review 17."},{"key":"e_1_3_3_82_2","doi-asserted-by":"crossref","first-page":"573","DOI":"10.1145\/3196709.3196729","volume-title":"Proceedings of the 2018 Designing Interactive Systems Conference","author":"Yang Qian","year":"2018","unstructured":"Qian Yang, Jina Suh, Nan-Chen Chen, and Gonzalo Ramos. 2018. Grounding interactive machine learning tool design in how non-experts actually build models. In Proceedings of the 2018 Designing Interactive Systems Conference. 573\u2013584."},{"key":"e_1_3_3_83_2","unstructured":"Good On You. 2023. Retrieved from https:\/\/goodonyou.eco\/"},{"key":"e_1_3_3_84_2","unstructured":"Xiaofei Zhou Jessica Van Brummelen and Phoebe Lin. 2020. Designing AI learning experiences for K-12: Emerging works future opportunities and a design framework. arXiv:2009.10228. Retrieved from https:\/\/arxiv.org\/abs\/cs\/2009.10228."},{"key":"e_1_3_3_85_2","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1145\/3392063.3394438","volume-title":"Proceedings of the Interaction Design and Children Conference","author":"Zimmermann-Niefield Abigail","year":"2020","unstructured":"Abigail Zimmermann-Niefield, Shawn Polson, Celeste Moreno, and R. Benjamin Shapiro. 2020. Youth making machine learning models for gesture-controlled interactive media. In Proceedings of the Interaction Design and Children Conference. 63\u201374."},{"key":"e_1_3_3_86_2","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1145\/3311927.3323139","volume-title":"Proceedings of the 18th ACM International Conference on Interaction Design and Children","author":"Zimmermann-Niefield Abigail","year":"2019","unstructured":"Abigail Zimmermann-Niefield, Makenna Turner, Bridget Murphy, Shaun K. Kane, and R. Benjamin Shapiro. 2019. Youth learning machine learning through building models of athletic moves. In Proceedings of the 18th ACM International Conference on Interaction Design and Children. 121\u2013132."}],"container-title":["ACM Transactions on Computing Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3641552","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3641552","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T18:50:16Z","timestamp":1750272616000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3641552"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,16]]},"references-count":85,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,6,30]]}},"alternative-id":["10.1145\/3641552"],"URL":"https:\/\/doi.org\/10.1145\/3641552","relation":{},"ISSN":["1946-6226"],"issn-type":[{"value":"1946-6226","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,16]]},"assertion":[{"value":"2023-06-29","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-01-08","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-04-16","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}