{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,24]],"date-time":"2026-04-24T17:48:24Z","timestamp":1777052904013,"version":"3.51.4"},"publisher-location":"New York, NY, USA","reference-count":31,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,2,26]],"date-time":"2020-02-26T00:00:00Z","timestamp":1582675200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1347722, 1150490"],"award-info":[{"award-number":["1347722, 1150490"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,2,26]]},"DOI":"10.1145\/3328778.3366879","type":"proceedings-article","created":{"date-parts":[[2020,2,25]],"date-time":"2020-02-25T09:02:03Z","timestamp":1582621323000},"page":"563-569","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":28,"title":["A Validated Scoring Rubric for Explain-in-Plain-English Questions"],"prefix":"10.1145","author":[{"given":"Binglin","family":"Chen","sequence":"first","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}]},{"given":"Sushmita","family":"Azad","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}]},{"given":"Rajarshi","family":"Haldar","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}]},{"given":"Matthew","family":"West","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}]},{"given":"Craig","family":"Zilles","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}]}],"member":"320","published-online":{"date-parts":[[2020,2,26]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/199688.199694"},{"key":"e_1_3_2_1_2_1","volume-title":"Evaluating the quality of learning: The SOLO taxonomy (Structure of the Observed Learning Outcome)","author":"Biggs John B","unstructured":"John B Biggs and Kevin F Collis . 2014. Evaluating the quality of learning: The SOLO taxonomy (Structure of the Observed Learning Outcome) . Academic Press . John B Biggs and Kevin F Collis. 2014. Evaluating the quality of learning: The SOLO taxonomy (Structure of the Observed Learning Outcome) .Academic Press."},{"key":"e_1_3_2_1_3_1","volume-title":"Validity, Discriminating Power, and Respondent Preferences. Acta psychologica","author":"Preston Carolyn C","year":"2000","unstructured":"Carolyn C Preston and Andrew Colman . 2000. Optimal Number of Response Categories in Rating Scales: Reliability , Validity, Discriminating Power, and Respondent Preferences. Acta psychologica , Vol. 104 (04 2000 ), 1--15. https:\/\/doi.org\/10.1016\/S0001--6918(99)00050--5 10.1016\/S0001--6918(99)00050--5 Carolyn C Preston and Andrew Colman. 2000. Optimal Number of Response Categories in Rating Scales: Reliability, Validity, Discriminating Power, and Respondent Preferences. Acta psychologica , Vol. 104 (04 2000), 1--15. https:\/\/doi.org\/10.1016\/S0001--6918(99)00050--5"},{"key":"e_1_3_2_1_4_1","volume-title":"Patterns and Pedagogy. In The Proceedings of the Thirtieth SIGCSE Technical Symposium on Computer Science Education (SIGCSE '99)","author":"Michael","unstructured":"Michael J. Clancy and Marcia C. Linn. 1999 . Patterns and Pedagogy. In The Proceedings of the Thirtieth SIGCSE Technical Symposium on Computer Science Education (SIGCSE '99) . ACM, New York, NY, USA, 37--42. https:\/\/doi.org\/10.1145\/299649.299673 10.1145\/299649.299673 Michael J. Clancy and Marcia C. Linn. 1999. Patterns and Pedagogy. In The Proceedings of the Thirtieth SIGCSE Technical Symposium on Computer Science Education (SIGCSE '99). ACM, New York, NY, USA, 37--42. https:\/\/doi.org\/10.1145\/299649.299673"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2538862.2538911"},{"key":"e_1_3_2_1_6_1","volume-title":"Proceedings of the Thirteenth Australasian Computing Education Conference -","volume":"114","author":"Corney Malcolm","year":"2011","unstructured":"Malcolm Corney , Raymond Lister , and Donna Teague . 2011 . Early Relational Reasoning and the Novice Programmer: Swapping As the \"Hello World\" of Relational Reasoning . In Proceedings of the Thirteenth Australasian Computing Education Conference - Volume 114 (ACE '11). Australian Computer Society, Inc., Darlinghurst, Australia, Australia, 95--104. http:\/\/dl.acm.org\/citation.cfm?id=2459936.2459948 Malcolm Corney, Raymond Lister, and Donna Teague. 2011. Early Relational Reasoning and the Novice Programmer: Swapping As the \"Hello World\" of Relational Reasoning. In Proceedings of the Thirteenth Australasian Computing Education Conference - Volume 114 (ACE '11). Australian Computer Society, Inc., Darlinghurst, Australia, Australia, 95--104. http:\/\/dl.acm.org\/citation.cfm?id=2459936.2459948"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1404520.1404532"},{"key":"e_1_3_2_1_8_1","volume-title":"Content analysis: An introduction to its methodology","author":"Krippendorff Klaus","unstructured":"Klaus Krippendorff . 2004. Content analysis: An introduction to its methodology . Sage publications. Klaus Krippendorff. 2004. Content analysis: An introduction to its methodology .Sage publications."},{"key":"e_1_3_2_1_9_1","article-title":"Multipoint Scales: Mean and Median Differences and Observed Significance","volume":"5","author":"Lewis James","year":"1993","unstructured":"James Lewis . 1993 . Multipoint Scales: Mean and Median Differences and Observed Significance Levels. Int. J. Hum. Comput. Interaction , Vol. 5 (10 1993), 383--392. https:\/\/doi.org\/10.1080\/10447319309526075 10.1080\/10447319309526075 James Lewis. 1993. Multipoint Scales: Mean and Median Differences and Observed Significance Levels. Int. J. Hum. Comput. Interaction , Vol. 5 (10 1993), 383--392. https:\/\/doi.org\/10.1080\/10447319309526075","journal-title":"Levels. Int. J. Hum. Comput. Interaction"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1207\/s15326985ep2004_4"},{"key":"e_1_3_2_1_11_1","volume-title":"Kate Sanders, Otto Sepp\"al\"a, Beth Simon, and Lynda Thomas.","author":"Lister Raymond","year":"2004","unstructured":"Raymond Lister , Elizabeth S Adams , Sue Fitzgerald , William Fone , John Hamer , Morten Lindholm , Robert McCartney , Jan Erik Mostr\u00f6m , Kate Sanders, Otto Sepp\"al\"a, Beth Simon, and Lynda Thomas. 2004 . A multi-national study of reading and tracing skills in novice programmers. ACM SIGCSE Bulletin , Vol. 36 , 4 (2004), 119--150. Raymond Lister, Elizabeth S Adams, Sue Fitzgerald, William Fone, John Hamer, Morten Lindholm, Robert McCartney, Jan Erik Mostr\u00f6m, Kate Sanders, Otto Sepp\"al\"a, Beth Simon, and Lynda Thomas. 2004. A multi-national study of reading and tracing skills in novice programmers. ACM SIGCSE Bulletin , Vol. 36, 4 (2004), 119--150."},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/1562877.1562930"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1140124.1140157"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/1404520.1404531"},{"key":"e_1_3_2_1_15_1","first-page":"2","article-title":"Natural Language Programming","volume":"20","author":"Miller L.","year":"1981","unstructured":"L. A.. Miller . 1981 . Natural Language Programming : Styles, Strategies, and Contrasts. IBM Syst. J. , Vol. 20 , 2 (June 1981), 184--215. https:\/\/doi.org\/10.1147\/sj.202.0184 10.1147\/sj.202.0184 L. A.. Miller. 1981. Natural Language Programming: Styles, Strategies, and Contrasts. IBM Syst. J. , Vol. 20, 2 (June 1981), 184--215. https:\/\/doi.org\/10.1147\/sj.202.0184","journal-title":"Styles, Strategies, and Contrasts. IBM Syst. J."},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2361276.2361299"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2157136.2157249"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3105726.3106178"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3286960.3286962"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1151588.1151594"},{"key":"e_1_3_2_1_21_1","article-title":"A Review of Generic Program Visualization Systems for Introductory Programming","volume":"13","author":"Sorva Juha","year":"2013","unstructured":"Juha Sorva , Ville Karavirta , and Lauri Malmi . 2013 . A Review of Generic Program Visualization Systems for Introductory Programming Education. Trans. Comput. Educ. , Vol. 13 , 4, Article 15 (Nov. 2013), bibinfonumpages64 pages. https:\/\/doi.org\/10.1145\/2490822 10.1145\/2490822 Juha Sorva, Ville Karavirta, and Lauri Malmi. 2013. A Review of Generic Program Visualization Systems for Introductory Programming Education. Trans. Comput. Educ. , Vol. 13, 4, Article 15 (Nov. 2013), bibinfonumpages64 pages. https:\/\/doi.org\/10.1145\/2490822","journal-title":"Education. Trans. Comput. Educ."},{"key":"e_1_3_2_1_22_1","volume-title":"Jeroen JG Van Merrienboer, and Fred GWC Paas","author":"Sweller John","year":"1998","unstructured":"John Sweller , Jeroen JG Van Merrienboer, and Fred GWC Paas . 1998 . Cognitive architecture and instructional design. Educational psychology review , Vol. 10 , 3 (1998), 251--296. John Sweller, Jeroen JG Van Merrienboer, and Fred GWC Paas. 1998. Cognitive architecture and instructional design. Educational psychology review , Vol. 10, 3 (1998), 251--296."},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.5116\/ijme.4dfb.8dfd"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1584322.1584336"},{"key":"e_1_3_2_1_25_1","volume-title":"Proceedings of the 2014 Conference on Innovation & Technology in Computer Science Education (ITiCSE '14)","author":"Watson Christopher","unstructured":"Christopher Watson and Frederick W.B. Li . 2014. Failure Rates in Introductory Programming Revisited . In Proceedings of the 2014 Conference on Innovation & Technology in Computer Science Education (ITiCSE '14) . ACM, New York, NY, USA, 39--44. https:\/\/doi.org\/10.1145\/2591708.2591749 10.1145\/2591708.2591749 Christopher Watson and Frederick W.B. Li. 2014. Failure Rates in Introductory Programming Revisited. In Proceedings of the 2014 Conference on Innovation & Technology in Computer Science Education (ITiCSE '14). ACM, New York, NY, USA, 39--44. https:\/\/doi.org\/10.1145\/2591708.2591749"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.18260\/p.24575"},{"key":"e_1_3_2_1_27_1","volume-title":"Eighth Australasian Computing Education Conference (ACE2006)","author":"Whalley Jacqueline","year":"2006","unstructured":"Jacqueline Whalley , Raymond Lister , Errol Thompson , Tony Clear , Phil Robbins , P K Ajith Kumar , and Christine Prasad . 2006 . An Australasian study of Reading and Comprehension Skills in Novice Programmers, using the Bloom and SOLO Taxonomies . Eighth Australasian Computing Education Conference (ACE2006) (01 2006). Jacqueline Whalley, Raymond Lister, Errol Thompson, Tony Clear, Phil Robbins, P K Ajith Kumar, and Christine Prasad. 2006. An Australasian study of Reading and Comprehension Skills in Novice Programmers, using the Bloom and SOLO Taxonomies . Eighth Australasian Computing Education Conference (ACE2006) (01 2006)."},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-7373(85)80041-9"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1080\/08993408.2019.1565235"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.5220\/0007753304140420"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/FIE.2018.8658551"}],"event":{"name":"SIGCSE '20: The 51st ACM Technical Symposium on Computer Science Education","location":"Portland OR USA","acronym":"SIGCSE '20","sponsor":["SIGCSE ACM Special Interest Group on Computer Science Education"]},"container-title":["Proceedings of the 51st ACM Technical Symposium on Computer Science Education"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3328778.3366879","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3328778.3366879","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3328778.3366879","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:13:05Z","timestamp":1750201985000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3328778.3366879"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,2,26]]},"references-count":31,"alternative-id":["10.1145\/3328778.3366879","10.1145\/3328778"],"URL":"https:\/\/doi.org\/10.1145\/3328778.3366879","relation":{},"subject":[],"published":{"date-parts":[[2020,2,26]]},"assertion":[{"value":"2020-02-26","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}