{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T20:27:40Z","timestamp":1776112060121,"version":"3.50.1"},"reference-count":66,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW2","license":[{"start":{"date-parts":[[2021,10,13]],"date-time":"2021-10-13T00:00:00Z","timestamp":1634083200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Science Foundation","award":["IIS 1755908"],"award-info":[{"award-number":["IIS 1755908"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2021,10,13]]},"abstract":"<jats:p>Peer assessment, as a form of collaborative learning, can engage students in active learning and improve their learning gains. However, current teaching platforms and programming environments provide little support to integrate peer assessment for in-class programming exercises. We identified challenges in conducting such exercises and adopting peer assessment through formative interviews with instructors of introductory programming courses. To address these challenges, we introduce PuzzleMe, a tool to help Computer Science instructors to conduct engaging in-class programming exercises. PuzzleMe leverages peer assessment to support a collaboration model where students provide timely feedback on their peers' work. We propose two assessment techniques tailored to in-class programming exercises: live peer testing and live peer code review. Live peer testing can improve students' code robustness by allowing them to create and share lightweight tests with peers. Live peer code review can improve code understanding by intelligently grouping students to maximize meaningful code reviews. A two-week deployment study revealed that PuzzleMe encourages students to write useful test cases, identify code problems, correct misunderstandings, and learn a diverse set of problem-solving approaches from peers.<\/jats:p>","DOI":"10.1145\/3479559","type":"journal-article","created":{"date-parts":[[2021,10,19]],"date-time":"2021-10-19T02:39:17Z","timestamp":1634611157000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":20,"title":["PuzzleMe: Leveraging Peer Assessment for In-Class Programming Exercises"],"prefix":"10.1145","volume":"5","author":[{"given":"April Yi","family":"Wang","sequence":"first","affiliation":[{"name":"University of Michigan, Ann Arbor, MI, USA"}]},{"given":"Yan","family":"Chen","sequence":"additional","affiliation":[{"name":"University of Michigan, Ann Arbor, MI, USA"}]},{"given":"John Joon Young","family":"Chung","sequence":"additional","affiliation":[{"name":"University of Michigan, Ann Arbor, MI, USA"}]},{"given":"Christopher","family":"Brooks","sequence":"additional","affiliation":[{"name":"University of Michigan, Ann Arbor, MI, USA"}]},{"given":"Steve","family":"Oney","sequence":"additional","affiliation":[{"name":"University of Michigan, Ann Arbor, MI, USA"}]}],"member":"320","published-online":{"date-parts":[[2021,10,18]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"How learning works: Seven research-based principles for smart teaching","author":"Ambrose Susan A","unstructured":"Susan A Ambrose, Michael W Bridges, Michele DiPietro, Marsha C Lovett, and Marie K Norman. 2010. How learning works: Seven research-based principles for smart teaching. John Wiley & Sons."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/6592.6593"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3328778.3366887"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2807442.2807473"},{"key":"e_1_2_1_5_1","volume-title":"Cocode: Co-learner Screen Sharing for Social Translucence in Online Programming Courses. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. 1--4.","author":"Byun Jeongmin","year":"2020","unstructured":"Jeongmin Byun, Jungkook Park, and Alice Oh. 2020. Cocode: Co-learner Screen Sharing for Social Translucence in Online Programming Courses. In Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. 1--4."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3173868"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3330430.3333627"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/VL\/HCC50065.2020.9127260"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3434168"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376835"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2713609.2713612"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-0663.88.4.715"},{"key":"e_1_2_1_13_1","volume-title":"Peer instruction: Ten years of experience and results. American journal of physics","author":"Crouch Catherine H","year":"2001","unstructured":"Catherine H Crouch and Eric Mazur. 2001. Peer instruction: Ten years of experience and results. American journal of physics, Vol. 69, 9 (2001), 970--977."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1080\/15391523.2014.888272"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1953163.1953299"},{"key":"e_1_2_1_16_1","volume-title":"https:\/\/github.com\/share\/sharedb Accessed","author":"Collaborative editing library. 2020. ShareDB.","year":"2020","unstructured":"Collaborative editing library. 2020. ShareDB. https:\/\/github.com\/share\/sharedb Accessed: April, 2020."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3059009.3059055"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380414"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2007.23"},{"key":"e_1_2_1_20_1","unstructured":"Jeremiah T Folsom-Kovarik Sae Schatz and Denise Nicholson. [n.d.]. Plan ahead: Pricing ITS learner models."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2807442.2807495"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818048.2820011"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2699751"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2047196.2047215"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.5555\/369272.357715"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3192366.3192387"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2807442.2807469"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/VLHCC.2015.7357201"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3051457.3051467"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858195"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2499947.2499951"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/1352135.1352315"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1121341.1121419"},{"key":"e_1_2_1_34_1","unstructured":"Juho Kim et al. 2015. Learnersourcing: improving learning with collective learner activity. Ph.D. Dissertation. Massachusetts Institute of Technology."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.learninstruc.2009.08.005"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2724660.2724676"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2505057"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2724660.2724670"},{"key":"e_1_2_1_39_1","doi-asserted-by":"crossref","unstructured":"Patrick R Lowenthal. 2010. Social presence. In Social computing: Concepts methodologies tools and applications. IGI global 129--136.","DOI":"10.4018\/978-1-60566-984-7.ch011"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/1352135.1352260"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2724660.2728672"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2998181.2998319"},{"key":"e_1_2_1_43_1","volume-title":"Proceedings of the ACM on Human-Computer Interaction","volume":"2","author":"Oney Steve","year":"2018","unstructured":"Steve Oney, Christopher Brooks, and Paul Resnick. 2018. Creating Guided Code Explanations with chat. codes. Proceedings of the ACM on Human-Computer Interaction, Vol. 2, CSCW (2018), 131."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386527.3405912"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/2839509.2844626"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/2632320.2632347"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.5555\/1047846.1047852"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376857"},{"key":"e_1_2_1_49_1","volume-title":"Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. 715--725","author":"Roman","year":"2017","unstructured":"Roman R\"adle, Midas Nouwens, Kristian Antonsen, James R Eagan, and Clemens N Klokmose. 2017. Codestrates: Literate computing with webstrates. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. 715--725."},{"key":"e_1_2_1_50_1","volume-title":"AIED Workshops .","author":"Rivers Kelly","unstructured":"Kelly Rivers and Kenneth R. Koedinger. 2013. Automatic Generation of Programming Feedback; A Data-Driven Approach. In AIED Workshops ."},{"key":"e_1_2_1_51_1","volume-title":"International Journal of Artificial Intelligence in Education","author":"Rivers Kelly","year":"2015","unstructured":"Kelly Rivers and Kenneth R Koedinger. 2015. Data-driven hint generation in vast solution spaces: a self-improving python programming tutor. International Journal of Artificial Intelligence in Education (2015), 1--28."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2017.68"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/2445196.2445388"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.24059\/olj.v24i1.1632"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/2491956.2462195"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICALT.2003.1215052"},{"key":"e_1_2_1_57_1","unstructured":"Barbara Leigh Smith and Jean T MacGregor. 1992. What is collaborative learning."},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/2361276.2361295"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1109\/FIE.2004.1408638"},{"key":"e_1_2_1_60_1","volume-title":"Proceedings of the Fifth Australasian Conference on Computing Education -","volume":"20","author":"Venables Anne","year":"2003","unstructured":"Anne Venables and Liz Haywood. 2003. Programming Students NEED Instant Feedback!. In Proceedings of the Fifth Australasian Conference on Computing Education - Volume 20 (Adelaide, Australia) (ACE '03). Australian Computer Society, Inc., AUS, 267--272."},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/2675133.2675219"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/2876034.2876042"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1080\/08993408.2019.1565235"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/3334480.3383068"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818048.2819953"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818048.2819953"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3479559","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3479559","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3479559","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T04:57:21Z","timestamp":1752469041000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3479559"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,13]]},"references-count":66,"journal-issue":{"issue":"CSCW2","published-print":{"date-parts":[[2021,10,13]]}},"alternative-id":["10.1145\/3479559"],"URL":"https:\/\/doi.org\/10.1145\/3479559","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,13]]},"assertion":[{"value":"2021-10-18","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}