{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,28]],"date-time":"2026-03-28T09:35:08Z","timestamp":1774690508302,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":33,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,2,2]],"date-time":"2021-02-02T00:00:00Z","timestamp":1612224000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,2,2]]},"DOI":"10.1145\/3441636.3442306","type":"proceedings-article","created":{"date-parts":[[2021,3,17]],"date-time":"2021-03-17T16:59:03Z","timestamp":1616000343000},"page":"66-72","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["The Impact of Multiple Choice Question Design on Predictions of Performance"],"prefix":"10.1145","author":[{"given":"Casey","family":"Wong","sequence":"first","affiliation":[{"name":"The University of Auckland, New Zealand"}]},{"given":"Paul","family":"Denny","sequence":"additional","affiliation":[{"name":"The University of Auckland, New Zealand"}]},{"given":"Andrew","family":"Luxton-Reilly","sequence":"additional","affiliation":[{"name":"The University of Auckland, New Zealand"}]},{"given":"Jacqueline","family":"Whalley","sequence":"additional","affiliation":[{"name":"Auckland University of Technology, New Zealand"}]}],"member":"320","published-online":{"date-parts":[[2021,3,17]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3243137"},{"key":"e_1_3_2_1_2_1","volume-title":"Taxonomy of educational objectives: Handbook I. Cognitive domain","author":"Bloom S","year":"1956","unstructured":"Benjamin\u00a0 S Bloom , MD Englehart , Edward\u00a0 J Furst , Walker\u00a0 H Hill , and David\u00a0 R Krathwohl . 1956. Taxonomy of educational objectives: Handbook I. Cognitive domain . New York : David McKay( 1956 ). Benjamin\u00a0S Bloom, MD Englehart, Edward\u00a0J Furst, Walker\u00a0H Hill, and David\u00a0R Krathwohl. 1956. Taxonomy of educational objectives: Handbook I. Cognitive domain. New York: David McKay(1956)."},{"key":"e_1_3_2_1_3_1","volume-title":"Team-based learning","author":"Brame J.","year":"2020","unstructured":"Cynthia\u00a0 J. Brame . 2020. Team-based learning . Vanderbilt University . Retrieved October 12, 2020 from https:\/\/cft.vanderbilt.edu\/guides-sub-pages\/team-based-learning\/ Cynthia\u00a0J. Brame. 2020. Team-based learning. Vanderbilt University. Retrieved October 12, 2020 from https:\/\/cft.vanderbilt.edu\/guides-sub-pages\/team-based-learning\/"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1080\/02602930410001689153"},{"key":"e_1_3_2_1_5_1","volume-title":"Proceedings of the Fourteenth Australasian Computing Education Conference-Volume 123","author":"Chinn Donald","year":"2012","unstructured":"Donald Chinn , Michael de Raadt , Anne Philpott , Judy Sheard , Mikko-Jussi Laakso , Daryl D\u2019Souza , James Skene , Angela Carbone , Tony Clear , Raymond Lister , 2012 . Introductory programming: examining the exams . In Proceedings of the Fourteenth Australasian Computing Education Conference-Volume 123 . 61\u201370. Donald Chinn, Michael de Raadt, Anne Philpott, Judy Sheard, Mikko-Jussi Laakso, Daryl D\u2019Souza, James Skene, Angela Carbone, Tony Clear, Raymond Lister, 2012. Introductory programming: examining the exams. In Proceedings of the Fourteenth Australasian Computing Education Conference-Volume 123. 61\u201370."},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.15388\/infedu.2004.12"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1322-7696(08)60478-3"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1404520.1404526"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/1822090.1822124"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287324.3287357"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3059009.3059033"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10459-004-4019-5"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF02294088"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197091.3197116"},{"key":"e_1_3_2_1_15_1","unstructured":"Nicole Ibbett and Brett Wheldon. 2016. The Incidence of Clueing in Multiple Choice Testbank Questions in Accounting: Some Evidence from Australia. e-Journal of Business Education and Scholarship of Teaching 10 1(2016) 20\u201335.  Nicole Ibbett and Brett Wheldon. 2016. The Incidence of Clueing in Multiple Choice Testbank Questions in Accounting: Some Evidence from Australia. e-Journal of Business Education and Scholarship of Teaching 10 1(2016) 20\u201335."},{"key":"e_1_3_2_1_16_1","volume-title":"Self-efficacy and academic achievement: Why do implicit beliefs, goals, and effort regulation matter?Learning and individual differences 25","author":"Komarraju Meera","year":"2013","unstructured":"Meera Komarraju and Dustin Nadler . 2013. Self-efficacy and academic achievement: Why do implicit beliefs, goals, and effort regulation matter?Learning and individual differences 25 ( 2013 ), 67\u201372. Meera Komarraju and Dustin Nadler. 2013. Self-efficacy and academic achievement: Why do implicit beliefs, goals, and effort regulation matter?Learning and individual differences 25 (2013), 67\u201372."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"crossref","unstructured":"Justin Kruger and David Dunning. 1999. Unskilled and unaware of it: how difficulties in recognizing one\u2019s own incompetence lead to inflated self-assessments.Journal of personality and social psychology 77 6(1999) 1121.  Justin Kruger and David Dunning. 1999. Unskilled and unaware of it: how difficulties in recognizing one\u2019s own incompetence lead to inflated self-assessments.Journal of personality and social psychology 77 6(1999) 1121.","DOI":"10.1037\/0022-3514.77.6.1121"},{"key":"e_1_3_2_1_18_1","volume-title":"Unskilled and Unaware of It: How Difficulties in Recognizing One\u2019s Own Incompetence Lead to Inflated Self-Assessments. Journal of Personality and Social Psychology 77 (01","author":"Kruger Justin","year":"2000","unstructured":"Justin Kruger and David Dunning . 2000. Unskilled and Unaware of It: How Difficulties in Recognizing One\u2019s Own Incompetence Lead to Inflated Self-Assessments. Journal of Personality and Social Psychology 77 (01 2000 ), 1121\u201334. https:\/\/doi.org\/10.1037\/\/0022-3514.77.6.1121 Justin Kruger and David Dunning. 2000. Unskilled and Unaware of It: How Difficulties in Recognizing One\u2019s Own Incompetence Lead to Inflated Self-Assessments. Journal of Personality and Social Psychology 77 (01 2000), 1121\u201334. https:\/\/doi.org\/10.1037\/\/0022-3514.77.6.1121"},{"key":"e_1_3_2_1_19_1","volume-title":"How well do multiple choice tests evaluate student understanding in computer programming classes?Journal of Information Systems Education 14, 4","author":"Kuechler L","year":"2003","unstructured":"William\u00a0 L Kuechler and Mark\u00a0 G Simkin . 2003. How well do multiple choice tests evaluate student understanding in computer programming classes?Journal of Information Systems Education 14, 4 ( 2003 ), 389. William\u00a0L Kuechler and Mark\u00a0G Simkin. 2003. How well do multiple choice tests evaluate student understanding in computer programming classes?Journal of Information Systems Education 14, 4 (2003), 389."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/359369.359393"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/364447.364605"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.crad.2007.08.021"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1080\/03098770601167922"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2899415.2925503"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1953163.1953340"},{"key":"e_1_3_2_1_26_1","volume-title":"Proceedings of the 8th Australasian Conference on Computing Education -","volume":"52","author":"Roberts S.","year":"2006","unstructured":"Tim\u00a0 S. Roberts . 2006 . The Use of Multiple Choice Tests for Formative and Summative Assessment . In Proceedings of the 8th Australasian Conference on Computing Education - Volume 52 (Hobart, Australia) (ACE \u201906). Australian Computer Society, Inc., AUS, 175\u2013180. Tim\u00a0S. Roberts. 2006. The Use of Multiple Choice Tests for Formative and Summative Assessment. In Proceedings of the 8th Australasian Conference on Computing Education - Volume 52 (Hobart, Australia) (ACE \u201906). Australian Computer Society, Inc., AUS, 175\u2013180."},{"key":"e_1_3_2_1_27_1","first-page":"123","article-title":"Does number of options in multiple choice tests affect item facility and discrimination? An examination of test-taker preferences","volume":"9","author":"Sadeghi Karim","year":"2017","unstructured":"Karim Sadeghi and Ghazal\u00a0Akhavan Masoumi . 2017 . Does number of options in multiple choice tests affect item facility and discrimination? An examination of test-taker preferences . Journal of English Language Teaching and Learning 9 , 19 (2017), 123 \u2013 143 . Karim Sadeghi and Ghazal\u00a0Akhavan Masoumi. 2017. Does number of options in multiple choice tests affect item facility and discrimination? An examination of test-taker preferences. Journal of English Language Teaching and Learning 9, 19 (2017), 123\u2013143.","journal-title":"Journal of English Language Teaching and Learning"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1080\/08993408.2010.509097"},{"key":"e_1_3_2_1_29_1","volume-title":"Proceedings of the Thirteenth Australasian Computing Education Conference -","volume":"114","year":"2011","unstructured":"Simon. 2011 . Wrong is a Relative Concept: Part Marks for Multiple-Choice Questions . In Proceedings of the Thirteenth Australasian Computing Education Conference - Volume 114 (Perth, Australia) (ACE \u201911). Australian Computer Society, Inc., AUS, 47\u201354. Simon. 2011. Wrong is a Relative Concept: Part Marks for Multiple-Choice Questions. In Proceedings of the Thirteenth Australasian Computing Education Conference - Volume 114 (Perth, Australia) (ACE \u201911). Australian Computer Society, Inc., AUS, 47\u201354."},{"key":"e_1_3_2_1_30_1","volume-title":"Proceedings of the 17th Australasian Computing Education Conference (ACE","author":"Sheard Judy","year":"2015","unstructured":"Simon, Judy Sheard , Daryl D\u2019Souza , Mike Lopez , Andrew Luxton-Reilly , Iwan\u00a0Handoyo Putro , Phil Robbins , Donna Teague , and Jacqueline Whalley . 2015 . How (not) to write an introductory programming exam . In Proceedings of the 17th Australasian Computing Education Conference (ACE 2015). Australian Computer Society, Inc., Sydney, Australia, 137\u2013146. Simon, Judy Sheard, Daryl D\u2019Souza, Mike Lopez, Andrew Luxton-Reilly, Iwan\u00a0Handoyo Putro, Phil Robbins, Donna Teague, and Jacqueline Whalley. 2015. How (not) to write an introductory programming exam. In Proceedings of the 17th Australasian Computing Education Conference (ACE 2015). Australian Computer Society, Inc., Sydney, Australia, 137\u2013146."},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2674683.2674701"},{"key":"e_1_3_2_1_32_1","volume-title":"Proceedings of the 7th Australasian Conference on Computing Education -","volume":"42","author":"Woodford Karyn","year":"2005","unstructured":"Karyn Woodford and Peter Bancroft . 2005 . Multiple Choice Questions Not Considered Harmful . In Proceedings of the 7th Australasian Conference on Computing Education - Volume 42 (Newcastle, New South Wales, Australia) (ACE \u201905). Australian Computer Society, Inc., AUS, 109\u2013116. Karyn Woodford and Peter Bancroft. 2005. Multiple Choice Questions Not Considered Harmful. In Proceedings of the 7th Australasian Conference on Computing Education - Volume 42 (Newcastle, New South Wales, Australia) (ACE \u201905). Australian Computer Society, Inc., AUS, 109\u2013116."},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1080\/00220671.1987.10885782"}],"event":{"name":"ACE '21: Australasian Computing Education Conference","location":"Virtual SA Australia","acronym":"ACE '21"},"container-title":["Proceedings of the 23rd Australasian Computing Education Conference"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3441636.3442306","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3441636.3442306","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:03:06Z","timestamp":1750197786000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3441636.3442306"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,2]]},"references-count":33,"alternative-id":["10.1145\/3441636.3442306","10.1145\/3441636"],"URL":"https:\/\/doi.org\/10.1145\/3441636.3442306","relation":{},"subject":[],"published":{"date-parts":[[2021,2,2]]},"assertion":[{"value":"2021-03-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}