{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,24]],"date-time":"2026-04-24T17:48:25Z","timestamp":1777052905277,"version":"3.51.4"},"publisher-location":"New York, NY, USA","reference-count":31,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,2,22]],"date-time":"2022-02-22T00:00:00Z","timestamp":1645488000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"NSF (National Science Foundation)","doi-asserted-by":"publisher","award":["DUE-1915257, DUE-2121424"],"award-info":[{"award-number":["DUE-1915257, DUE-2121424"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,2,22]]},"DOI":"10.1145\/3478431.3499409","type":"proceedings-article","created":{"date-parts":[[2022,2,23]],"date-time":"2022-02-23T00:53:28Z","timestamp":1645577608000},"page":"133-139","source":"Crossref","is-referenced-by-count":8,"title":["Peer-grading \"Explain in Plain English\""],"prefix":"10.1145","author":[{"given":"Binglin","family":"Chen","sequence":"first","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}]},{"given":"Matthew","family":"West","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}]},{"given":"Craig","family":"Zilles","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}]}],"member":"320","published-online":{"date-parts":[[2022,2,22]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/199688.199694"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3328778.3366879"},{"key":"e_1_3_2_1_3_1","volume-title":"Patterns and Pedagogy. In The Proceedings of the Thirtieth SIGCSE Technical Symposium on Computer Science Education","author":"Michael","unstructured":"Michael J. Clancy and Marcia C. Linn. 1999 . Patterns and Pedagogy. In The Proceedings of the Thirtieth SIGCSE Technical Symposium on Computer Science Education ( New Orleans, Louisiana, USA) (SIGCSE '99). ACM, New York, NY, USA, 37--42. https:\/\/doi.org\/10.1145\/299649.299673 Michael J. Clancy and Marcia C. Linn. 1999. Patterns and Pedagogy. In The Proceedings of the Thirtieth SIGCSE Technical Symposium on Computer Science Education (New Orleans, Louisiana, USA) (SIGCSE '99). ACM, New York, NY, USA, 37--42. https:\/\/doi.org\/10.1145\/299649.299673"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2538862.2538911"},{"key":"e_1_3_2_1_5_1","volume-title":"Proceedings of the Thirteenth Australasian Computing Education Conference -","volume":"114","author":"Corney Malcolm","year":"2011","unstructured":"Malcolm Corney , Raymond Lister , and Donna Teague . 2011 . Early Relational Reasoning and the Novice Programmer: Swapping As the \"Hello World\" of Relational Reasoning . In Proceedings of the Thirteenth Australasian Computing Education Conference - Volume 114 (Perth, Australia) (ACE '11). Australian Computer Society, Inc., Darlinghurst, Australia, Australia, 95--104. http:\/\/dl.acm.org\/citation.cfm?id=2459936.2459948 Malcolm Corney, Raymond Lister, and Donna Teague. 2011. Early Relational Reasoning and the Novice Programmer: Swapping As the \"Hello World\" of Relational Reasoning. In Proceedings of the Thirteenth Australasian Computing Education Conference - Volume 114 (Perth, Australia) (ACE '11). Australian Computer Society, Inc., Darlinghurst, Australia, Australia, 95--104. http:\/\/dl.acm.org\/citation.cfm?id=2459936.2459948"},{"key":"e_1_3_2_1_6_1","volume-title":"Student peer assessment in higher education: A meta-analysis comparing peer and teacher marks. Review of educational research","author":"Falchikov Nancy","year":"2000","unstructured":"Nancy Falchikov and Judy Goldfinch . 2000. Student peer assessment in higher education: A meta-analysis comparing peer and teacher marks. Review of educational research , Vol. 70 , 3 ( 2000 ), 287--322. Nancy Falchikov and Judy Goldfinch. 2000. Student peer assessment in higher education: A meta-analysis comparing peer and teacher marks. Review of educational research , Vol. 70, 3 (2000), 287--322."},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3446871.3469738"},{"key":"e_1_3_2_1_8_1","volume-title":"Steve Croker, Peter CH Cheng, Gary Jones, Iain Oliver, and Julian M Pine.","author":"Gobet Fernand","year":"2001","unstructured":"Fernand Gobet , Peter CR Lane , Steve Croker, Peter CH Cheng, Gary Jones, Iain Oliver, and Julian M Pine. 2001 . Chunking mechanisms in human learning. Trends in cognitive sciences , Vol. 5 , 6 (2001), 236--243. Fernand Gobet, Peter CR Lane, Steve Croker, Peter CH Cheng, Gary Jones, Iain Oliver, and Julian M Pine. 2001. Chunking mechanisms in human learning. Trends in cognitive sciences , Vol. 5, 6 (2001), 236--243."},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.5555\/1082424.1082433"},{"key":"e_1_3_2_1_10_1","volume-title":"Kate Sanders, Otto Sepp\"al\"a, Beth Simon, and Lynda Thomas.","author":"Lister Raymond","year":"2004","unstructured":"Raymond Lister , Elizabeth S Adams , Sue Fitzgerald , William Fone , John Hamer , Morten Lindholm , Robert McCartney , Jan Erik Mostr\u00f6m , Kate Sanders, Otto Sepp\"al\"a, Beth Simon, and Lynda Thomas. 2004 . A multi-national study of reading and tracing skills in novice programmers. ACM SIGCSE Bulletin , Vol. 36 , 4 (2004), 119--150. Raymond Lister, Elizabeth S Adams, Sue Fitzgerald, William Fone, John Hamer, Morten Lindholm, Robert McCartney, Jan Erik Mostr\u00f6m, Kate Sanders, Otto Sepp\"al\"a, Beth Simon, and Lynda Thomas. 2004. A multi-national study of reading and tracing skills in novice programmers. ACM SIGCSE Bulletin , Vol. 36, 4 (2004), 119--150."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1562877.1562930"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/1404520.1404531"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1404520.1404531"},{"key":"e_1_3_2_1_14_1","article-title":"Peer grading in a MOOC: Reliability, validity, and perceived effects","volume":"18","author":"Luo Heng","year":"2014","unstructured":"Heng Luo , Anthony Robinson , and Jae-Young Park . 2014 . Peer grading in a MOOC: Reliability, validity, and perceived effects . Online Learning Journal , Vol. 18 , 2 (2014). Heng Luo, Anthony Robinson, and Jae-Young Park. 2014. Peer grading in a MOOC: Reliability, validity, and perceived effects. Online Learning Journal , Vol. 18, 2 (2014).","journal-title":"Online Learning Journal"},{"key":"e_1_3_2_1_15_1","volume-title":"Schemas in problem solving","author":"Marshall Sandra P","unstructured":"Sandra P Marshall . 1995. Schemas in problem solving . Cambridge University Press . Sandra P Marshall. 1995. Schemas in problem solving .Cambridge University Press."},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/0010-0285(81)90012-8"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2157136.2157249"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2729.2008.00290.x"},{"key":"e_1_3_2_1_19_1","volume-title":"Proceedings of the 6th International Conference on Educational Data Mining. 153--160","author":"Piech Chris","year":"2013","unstructured":"Chris Piech , Jonathan Huang , Zhenghao Chen , Chuong B. Do , Andrew Y. Ng , and Daphne Koller . 2013 . Tuned Models of Peer Assessment in MOOCs .. In Proceedings of the 6th International Conference on Educational Data Mining. 153--160 . Chris Piech, Jonathan Huang, Zhenghao Chen, Chuong B. Do, Andrew Y. Ng, and Daphne Koller. 2013. Tuned Models of Peer Assessment in MOOCs.. In Proceedings of the 6th International Conference on Educational Data Mining. 153--160."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevPhysEducRes.12.020145"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2623330.2623654"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"crossref","first-page":"474","DOI":"10.2307\/4451167","article-title":"Calibrated Peer Review? an application to increase student reading & writing skills","volume":"63","author":"Robinson Ralph","year":"2001","unstructured":"Ralph Robinson . 2001 . Calibrated Peer Review? an application to increase student reading & writing skills . The American Biology Teacher , Vol. 63 , 7 (2001), 474 -- 480 . Ralph Robinson. 2001. Calibrated Peer Review? an application to increase student reading & writing skills. The American Biology Teacher , Vol. 63, 7 (2001), 474--480.","journal-title":"The American Biology Teacher"},{"key":"e_1_3_2_1_23_1","volume-title":"The impact of self-and peer-grading on student learning. Educational assessment","author":"Sadler Philip M","year":"2006","unstructured":"Philip M Sadler and Eddie Good . 2006. The impact of self-and peer-grading on student learning. Educational assessment , Vol. 11 , 1 ( 2006 ), 1--31. Philip M Sadler and Eddie Good. 2006. The impact of self-and peer-grading on student learning. Educational assessment , Vol. 11, 1 (2006), 1--31."},{"key":"e_1_3_2_1_24_1","volume-title":"Psychology of learning and motivation .","author":"Sweller John","unstructured":"John Sweller . 2011. Cognitive Load Theory . In Psychology of learning and motivation . Vol. 55 . Elsevier , 37--76. John Sweller. 2011. Cognitive Load Theory . In Psychology of learning and motivation . Vol. 55. Elsevier, 37--76."},{"key":"e_1_3_2_1_25_1","volume-title":"Jeroen JG Van Merrienboer, and Fred GWC Paas","author":"Sweller John","year":"1998","unstructured":"John Sweller , Jeroen JG Van Merrienboer, and Fred GWC Paas . 1998 . Cognitive architecture and instructional design. Educational psychology review , Vol. 10 , 3 (1998), 251--296. John Sweller, Jeroen JG Van Merrienboer, and Fred GWC Paas. 1998. Cognitive architecture and instructional design. Educational psychology review , Vol. 10, 3 (1998), 251--296."},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1584322.1584336"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2724660.2724672"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.18260\/p.24575"},{"key":"e_1_3_2_1_29_1","volume-title":"Eighth Australasian Computing Education Conference (ACE2006)","author":"Whalley Jacqueline","year":"2006","unstructured":"Jacqueline Whalley , Raymond Lister , Errol Thompson , Tony Clear , Phil Robbins , P K Ajith Kumar , and Christine Prasad . 2006 . An Australasian study of Reading and Comprehension Skills in Novice Programmers, using the Bloom and SOLO Taxonomies . Eighth Australasian Computing Education Conference (ACE2006) (01 2006). Jacqueline Whalley, Raymond Lister, Errol Thompson, Tony Clear, Phil Robbins, P K Ajith Kumar, and Christine Prasad. 2006. An Australasian study of Reading and Comprehension Skills in Novice Programmers, using the Bloom and SOLO Taxonomies . Eighth Australasian Computing Education Conference (ACE2006) (01 2006)."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-7373(85)80041-9"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/234867.234872"}],"event":{"name":"SIGCSE 2022: The 53rd ACM Technical Symposium on Computer Science Education","location":"Providence RI USA","acronym":"SIGCSE 2022","sponsor":["SIGCSE ACM Special Interest Group on Computer Science Education"]},"container-title":["Proceedings of the 53rd ACM Technical Symposium on Computer Science Education"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3478431.3499409","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3478431.3499409","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3478431.3499409","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:31:15Z","timestamp":1750188675000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3478431.3499409"}},"subtitle":["A Bayesian Calibration Method for Categorical Answers"],"short-title":[],"issued":{"date-parts":[[2022,2,22]]},"references-count":31,"alternative-id":["10.1145\/3478431.3499409","10.1145\/3478431"],"URL":"https:\/\/doi.org\/10.1145\/3478431.3499409","relation":{},"subject":[],"published":{"date-parts":[[2022,2,22]]}}}