{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:31:20Z","timestamp":1750221080135,"version":"3.41.0"},"reference-count":52,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2018,11,19]],"date-time":"2018-11-19T00:00:00Z","timestamp":1542585600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Educ."],"published-print":{"date-parts":[[2019,3,31]]},"abstract":"<jats:p>Low performance of nontechnical engineering students in programming courses is a problem that remains unsolved. Over the years, many authors have tried to identify the multiple causes for that failure, but there is unanimity on the fact that motivation is a key factor for the acquisition of knowledge by students.<\/jats:p>\n          <jats:p>To better understand motivation, a new evaluation strategy has been adopted in a second programming course of a nontechnical degree, consisting of 91 students. The goals of the study were to identify if those students felt more motivated to answer multiple-choice questions in comparison to development questions, and what type of question better allows for testing student knowledge acquisition.<\/jats:p>\n          <jats:p>Possibilities around the motivational qualities of multiple-choice questions in programming courses will be discussed in light of the results.<\/jats:p>\n          <jats:p>In conclusion, it seems clear that student performance varies according to the type of question. Our study points out that multiple-choice questions can be seen as a motivational factor for engineering students and it might also be a good way to test acquired programming concepts. Therefore, this type of question could be further explored in the evaluation points.<\/jats:p>","DOI":"10.1145\/3243137","type":"journal-article","created":{"date-parts":[[2018,11,20]],"date-time":"2018-11-20T14:19:23Z","timestamp":1542723563000},"page":"1-16","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["Multiple-Choice Questions in Programming Courses"],"prefix":"10.1145","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9278-8194","authenticated-orcid":false,"given":"Pedro Henriques","family":"Abreu","sequence":"first","affiliation":[{"name":"Faculty of Sciences and Technology of the University of Coimbra, Coimbra, Portugal"}]},{"given":"Daniel Castro","family":"Silva","sequence":"additional","affiliation":[{"name":"Faculty of Engineering of the University of Porto, Portugal"}]},{"given":"Anabela","family":"Gomes","sequence":"additional","affiliation":[{"name":"Coimbra Institute of Engineering, Coimbra, Portugal"}]}],"member":"320","published-online":{"date-parts":[[2018,11,19]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.15388\/infedu.2016.01"},{"volume-title":"Proceedings of the Annual Academic Sessions of the Open University of Sri Lanka. 8--10","author":"Bandarage Gunadya","key":"e_1_2_1_2_1","unstructured":"Gunadya Bandarage , Marie N. Kumudinee de Zoysa, and Lankani Priyangika Wijesinghe. 2009. Effect of continuous assessment tests with multiple choice questions on motivation for meaningful learning . In Proceedings of the Annual Academic Sessions of the Open University of Sri Lanka. 8--10 . Gunadya Bandarage, Marie N. Kumudinee de Zoysa, and Lankani Priyangika Wijesinghe. 2009. Effect of continuous assessment tests with multiple choice questions on motivation for meaningful learning. In Proceedings of the Annual Academic Sessions of the Open University of Sri Lanka. 8--10."},{"volume-title":"Taxonomy of Educational Objectives: Book 1 Cognitive Domain","author":"Bloom Benjamin S.","key":"e_1_2_1_3_1","unstructured":"Benjamin S. Bloom . 1956. Taxonomy of Educational Objectives: Book 1 Cognitive Domain . Longman, New York . Benjamin S. Bloom. 1956. Taxonomy of Educational Objectives: Book 1 Cognitive Domain. Longman, New York."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1745-3984.1994.tb00433.x"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2839509.2844559"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.15388\/infedu.2004.12"},{"key":"e_1_2_1_7_1","volume-title":"ANZMAC 2005 Conference: Marketing Education. 51--57","author":"Clarke Peter David","year":"2005","unstructured":"Peter David Clarke , Joo-Gim Heaney , and Terry John Gatfield . 2005 . Multiple choice testing: A preferred assessment procedure that is fair to all our business students? In ANZMAC 2005 Conference: Marketing Education. 51--57 . Peter David Clarke, Joo-Gim Heaney, and Terry John Gatfield. 2005. Multiple choice testing: A preferred assessment procedure that is fair to all our business students? In ANZMAC 2005 Conference: Marketing Education. 51--57."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1480828.1480839"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2013.12.013"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.paid.2010.09.040"},{"volume-title":"Proceedings of the International Conference on Frontiers in Education. 8 pages.","author":"Gomes Anabela","key":"e_1_2_1_11_1","unstructured":"Anabela Gomes , Fernanda B. Correia , and Pedro H. Abreu . 2016. Types of assessing student-programming knowledge . In Proceedings of the International Conference on Frontiers in Education. 8 pages. Anabela Gomes, Fernanda B. Correia, and Pedro H. Abreu. 2016. Types of assessing student-programming knowledge. In Proceedings of the International Conference on Frontiers in Education. 8 pages."},{"volume-title":"Proceedings of the International Conference on Engineering Education. 5 pages.","author":"Gomes Anabela","key":"e_1_2_1_12_1","unstructured":"Anabela Gomes and Ant\u00f3nio J. Mendes . 2007. Learning to program - Difficulties and solutions . In Proceedings of the International Conference on Engineering Education. 5 pages. Anabela Gomes and Ant\u00f3nio J. Mendes. 2007. Learning to program - Difficulties and solutions. In Proceedings of the International Conference on Engineering Education. 5 pages."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2617658"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1080\/08832329709601623"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1080\/09500693.2014.916831"},{"volume-title":"HERDSA Green Guide No 16","author":"Isaacs Geoff","key":"e_1_2_1_16_1","unstructured":"Geoff Isaacs . 1994. Multiple choice testing , HERDSA Green Guide No 16 . Higher Education Research and Development Society of Australasia . Geoff Isaacs. 1994. Multiple choice testing, HERDSA Green Guide No 16. Higher Education Research and Development Society of Australasia."},{"volume-title":"The Motivation of Students of Programming. Master\u2019s thesis","author":"Jenkins Tony","key":"e_1_2_1_17_1","unstructured":"Tony Jenkins . 2001. The Motivation of Students of Programming. Master\u2019s thesis . University of Kent at Canterbury . Tony Jenkins. 2001. The Motivation of Students of Programming. Master\u2019s thesis. University of Kent at Canterbury."},{"key":"e_1_2_1_18_1","volume-title":"Strategies for stimulating the motivation to learn. Performance 8 Instruction 26, 8","author":"Keller John M.","year":"1987","unstructured":"John M. Keller . 1987. Strategies for stimulating the motivation to learn. Performance 8 Instruction 26, 8 ( 1987 ), 1--7. John M. Keller. 1987. Strategies for stimulating the motivation to learn. Performance 8 Instruction 26, 8 (1987), 1--7."},{"key":"e_1_2_1_19_1","volume-title":"The systematic process of motivational design. Performance 8 Instruction 26, 9","author":"Keller John M.","year":"1987","unstructured":"John M. Keller . 1987. The systematic process of motivational design. Performance 8 Instruction 26, 9 ( 1987 ), 1--8. John M. Keller. 1987. The systematic process of motivational design. Performance 8 Instruction 26, 9 (1987), 1--8."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.sbspro.2013.10.414"},{"key":"e_1_2_1_21_1","first-page":"389","article-title":"How well do multiple choice tests evaluate student understanding in computer programming classes","volume":"14","author":"Kuechler William L.","year":"2003","unstructured":"William L. Kuechler and Mark G. Simkin . 2003 . How well do multiple choice tests evaluate student understanding in computer programming classes . Journal of Information Systems Education 14 , 4 (2003), 389 -- 399 . William L. Kuechler and Mark G. Simkin. 2003. How well do multiple choice tests evaluate student understanding in computer programming classes. Journal of Information Systems Education 14, 4 (2003), 389--399.","journal-title":"Journal of Information Systems Education"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2785807"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2910925.2910950"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1227310.1227352"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/359369.359393"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/366413.364605"},{"key":"e_1_2_1_27_1","volume-title":"7th Australasian Computing Education Conference. 155--164","author":"Lister Raymond","year":"2005","unstructured":"Raymond Lister . 2005 . One small step toward a culture of peer review and multi-institutional sharing of educational resources: A multiple choice exam for first semester programming students . In 7th Australasian Computing Education Conference. 155--164 . Raymond Lister. 2005. One small step toward a culture of peer review and multi-institutional sharing of educational resources: A multiple choice exam for first semester programming students. In 7th Australasian Computing Education Conference. 155--164."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1041624.1041673"},{"key":"e_1_2_1_29_1","first-page":"1","article-title":"Cheating on multiple choice exams is difficult to assess quantitatively","volume":"30","author":"Marx D. B.","year":"1986","unstructured":"D. B. Marx and David E. Longer . 1986 . Cheating on multiple choice exams is difficult to assess quantitatively . NACTA Journal 30 , 1 (March 1986), 23--26. D. B. Marx and David E. Longer. 1986. Cheating on multiple choice exams is difficult to assess quantitatively. NACTA Journal 30, 1 (March 1986), 23--26.","journal-title":"NACTA Journal"},{"volume-title":"Innovative Techniques in Instruction Technology, E-Learning, E-Assessment","author":"Mow Ioana T.","key":"e_1_2_1_30_1","unstructured":"Ioana T. Mow . 2008. Issues and difficulties in teaching novice computer programming . In Innovative Techniques in Instruction Technology, E-Learning, E-Assessment , and Education, Magued Iskander (Ed.). Springer Netherlands , 199--204. Ioana T. Mow. 2008. Issues and difficulties in teaching novice computer programming. In Innovative Techniques in Instruction Technology, E-Learning, E-Assessment, and Education, Magued Iskander (Ed.). Springer Netherlands, 199--204."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1080\/03098770601167922"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1080\/03075070600572090"},{"key":"e_1_2_1_33_1","unstructured":"Victoria University of Wellington. 2013. A guide for developing multiple choice and other objective style questions. CAD Guidelines. Retrieved from http:\/\/www.victoria.ac.nz\/learning-teaching\/support\/approach\/guides\/developing-questions2\/developing-questions.pdf.  Victoria University of Wellington. 2013. A guide for developing multiple choice and other objective style questions. CAD Guidelines. Retrieved from http:\/\/www.victoria.ac.nz\/learning-teaching\/support\/approach\/guides\/developing-questions2\/developing-questions.pdf."},{"volume-title":"What are we doing when we assess programming? In 17th Australasian Computing Education. 119--127","author":"Parsons Dale","key":"e_1_2_1_34_1","unstructured":"Dale Parsons , Krissi Wood , and Patricia Haden . 2015. What are we doing when we assess programming? In 17th Australasian Computing Education. 119--127 . Dale Parsons, Krissi Wood, and Patricia Haden. 2015. What are we doing when we assess programming? In 17th Australasian Computing Education. 119--127."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2899415.2925503"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF02744525"},{"key":"e_1_2_1_37_1","volume-title":"8th Australasian Computing Education Conference. 175--180","author":"Roberts Tim","year":"2006","unstructured":"Tim Roberts . 2006 . The use of multiple choice tests for formative and summative assessment . In 8th Australasian Computing Education Conference. 175--180 . Tim Roberts. 2006. The use of multiple choice tests for formative and summative assessment. In 8th Australasian Computing Education Conference. 175--180."},{"key":"e_1_2_1_38_1","first-page":"1363","article-title":"Consequences the extensive use of multiple-choice questions might have on student\u2019s reasoning structure","volume":"58","author":"R\u01cedu\u0163\u01ce Cristian M.","year":"2013","unstructured":"Cristian M. R\u01cedu\u0163\u01ce . 2013 . Consequences the extensive use of multiple-choice questions might have on student\u2019s reasoning structure . Romain Journal of Phisics 58 , 9 (2013), 1363 -- 1380 . Cristian M. R\u01cedu\u0163\u01ce. 2013. Consequences the extensive use of multiple-choice questions might have on student\u2019s reasoning structure. Romain Journal of Phisics 58, 9 (2013), 1363--1380.","journal-title":"Romain Journal of Phisics"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.2991\/icaet-14.2014.22"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2013.04.030"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1080\/08993408.2010.509097"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1540-4609.2005.00053.x"},{"key":"e_1_2_1_43_1","volume-title":"14th Australasian Computing Education Conference. 61--70","author":"Sheard Judy","year":"2012","unstructured":"Simon, Judy Sheard , Angela Carbone , Donald Chinn , Mikko-Jussi Laakso , Tony Clear , Michael de Raadt , Daryl D\u2019Souza , Raymond Lister , Anne Philpott , James Skene , and Geoff Warburton . 2012 . Introductory programming: Examining the exams . In 14th Australasian Computing Education Conference. 61--70 . Simon, Judy Sheard, Angela Carbone, Donald Chinn, Mikko-Jussi Laakso, Tony Clear, Michael de Raadt, Daryl D\u2019Souza, Raymond Lister, Anne Philpott, James Skene, and Geoff Warburton. 2012. Introductory programming: Examining the exams. In 14th Australasian Computing Education Conference. 61--70."},{"key":"e_1_2_1_44_1","volume-title":"Proceedings of the 17th International Australasian Computing Education Conference. 137--146","author":"Sheared Judy","year":"2015","unstructured":"Simon, Judy Sheared , Daryl D\u2019Souza , Mike Lopez , Andrew Luxton-Reilly , Iwan H. Putro , Phil Robbins , Donna Teague , and Jacqueline Whalley . 2015 . How (not) to write an introductory programming exam . In Proceedings of the 17th International Australasian Computing Education Conference. 137--146 . Simon, Judy Sheared, Daryl D\u2019Souza, Mike Lopez, Andrew Luxton-Reilly, Iwan H. Putro, Phil Robbins, Donna Teague, and Jacqueline Whalley. 2015. How (not) to write an introductory programming exam. In Proceedings of the 17th International Australasian Computing Education Conference. 137--146."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/2674683.2674701"},{"volume-title":"Free-Response in the Testing of Scholastic Achievement","author":"Traub Ross E.","key":"e_1_2_1_46_1","unstructured":"Ross E. Traub . 1990. Multiple-Choice Vs . Free-Response in the Testing of Scholastic Achievement . Ontario Institute for Studies in Education. Ross E. Traub. 1990. Multiple-Choice Vs. Free-Response in the Testing of Scholastic Achievement. Ontario Institute for Studies in Education."},{"key":"e_1_2_1_47_1","volume-title":"Student perceptions of continuous summative assessment. Assessment 8 Evaluation in Higher Education 31, 5","author":"Trotter Eileen","year":"2006","unstructured":"Eileen Trotter . 2006. Student perceptions of continuous summative assessment. Assessment 8 Evaluation in Higher Education 31, 5 ( 2006 ), 505--521. Eileen Trotter. 2006. Student perceptions of continuous summative assessment. Assessment 8 Evaluation in Higher Education 31, 5 (2006), 505--521."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.2190\/4NK5-17L7-TWQV-1EHL"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2015.10.016"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1080\/03043797.2015.1056105"},{"key":"e_1_2_1_51_1","volume-title":"7th Australasian Computing Education Conference (ACE\u201905)","author":"Woodford Karyn","year":"2005","unstructured":"Karyn Woodford and Peter Bancroft . 2005 . Multiple choice questions not considered harmful . In 7th Australasian Computing Education Conference (ACE\u201905) . 109--115. Karyn Woodford and Peter Bancroft. 2005. Multiple choice questions not considered harmful. In 7th Australasian Computing Education Conference (ACE\u201905). 109--115."},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1080\/00220671.1987.10885782"}],"container-title":["ACM Transactions on Computing Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3243137","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3243137","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:57:39Z","timestamp":1750208259000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3243137"}},"subtitle":["Can We Use Them and Are Students Motivated by Them?"],"short-title":[],"issued":{"date-parts":[[2018,11,19]]},"references-count":52,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2019,3,31]]}},"alternative-id":["10.1145\/3243137"],"URL":"https:\/\/doi.org\/10.1145\/3243137","relation":{},"ISSN":["1946-6226"],"issn-type":[{"type":"electronic","value":"1946-6226"}],"subject":[],"published":{"date-parts":[[2018,11,19]]},"assertion":[{"value":"2017-08-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-07-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-11-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}