{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,18]],"date-time":"2026-05-18T16:23:14Z","timestamp":1779121394572,"version":"3.51.4"},"reference-count":47,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2017,8,28]],"date-time":"2017-08-28T00:00:00Z","timestamp":1503878400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Educ."],"published-print":{"date-parts":[[2017,9,30]]},"abstract":"<jats:p>We describe a method for analyzing student data from online programming exercises. Our approach uses contingency tables that combine whether or not a student answered an online exercise correctly with the number of attempts that the student made on that exercise. We use this method to explore the relationship between student performance on online exercises done during semester with subsequent performance on questions in a paper-based exam at the end of semester. We found that it is useful to include data about the number of attempts a student makes on an online exercise.<\/jats:p>","DOI":"10.1145\/3123814","type":"journal-article","created":{"date-parts":[[2017,8,28]],"date-time":"2017-08-28T12:45:15Z","timestamp":1503924315000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["A Contingency Table Derived Method for Analyzing Course Data"],"prefix":"10.1145","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2018-3065","authenticated-orcid":false,"given":"Alireza","family":"Ahadi","sequence":"first","affiliation":[{"name":"University of Technology, Sydney, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Arto","family":"Hellas","sequence":"additional","affiliation":[{"name":"University of Helsinki, Helsinki, Finland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Raymond","family":"Lister","sequence":"additional","affiliation":[{"name":"University of Technology, Sydney, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2017,8,28]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2839509.2844640"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2787622.2787717"},{"key":"e_1_2_1_3_1","doi-asserted-by":"crossref","unstructured":"D. G. Altman and J. M. Bland. 1994. Diagnostic tests. 1: Sensitivity and specificity. BMJ 308 6943 (1994) 1552.  D. G. Altman and J. M. Bland. 1994. Diagnostic tests. 1: Sensitivity and specificity. BMJ 308 6943 (1994) 1552.","DOI":"10.1136\/bmj.308.6943.1552"},{"key":"e_1_2_1_4_1","unstructured":"Sadaf Fatima Salim Attar and Y. C. Kulkarni. 2015. Precognition of students academic failure using data mining techniques. Int. J. Adv. Res. Comput. Commun. Eng. (2015).  Sadaf Fatima Salim Attar and Y. C. Kulkarni. 2015. Precognition of students academic failure using data mining techniques. Int. J. Adv. Res. Comput. Commun. Eng. (2015)."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/16.5.412"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/800038.801037"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1138403.1138430"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1047124.1047480"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1080\/08993400600997096"},{"key":"e_1_2_1_10_1","volume-title":"Proceedings of the Meeting of the American Educational Research Association","author":"Berland M.","unstructured":"M. Berland and T. Martin . 2011. Clusters and patterns of novice programmers . In Proceedings of the Meeting of the American Educational Research Association . New Orleans, LA. M. Berland and T. Martin. 2011. Clusters and patterns of novice programmers. In Proceedings of the Meeting of the American Educational Research Association. New Orleans, LA."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1080\/10508406.2013.836655"},{"key":"e_1_2_1_12_1","first-page":"99","article-title":"International vocabulary of metrology\u2014Basic and general concepts and associated terms, 2008","volume":"200","author":"IEC BiPM, ILAC IFCC, IUPAC","year":"2008","unstructured":"IEC BiPM, ILAC IFCC, IUPAC ISO, and OIML IUPAP. 2008 . International vocabulary of metrology\u2014Basic and general concepts and associated terms, 2008 . JCGM 200 (2008), 99 -- 12 . IEC BiPM, ILAC IFCC, IUPAC ISO, and OIML IUPAP. 2008. International vocabulary of metrology\u2014Basic and general concepts and associated terms, 2008. JCGM 200 (2008), 99--12.","journal-title":"JCGM"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2090116.2090132"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/377435.377467"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/364447.364581"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2787622.2787710"},{"key":"e_1_2_1_17_1","first-page":"156","article-title":"Knowledge mining from student data","volume":"47","author":"Chandra E.","year":"2010","unstructured":"E. Chandra and K. Nandhini . 2010 . Knowledge mining from student data . Eur. J. Sci. Res. 47 , 1 (2010), 156 -- 163 . E. Chandra and K. Nandhini. 2010. Knowledge mining from student data. Eur. J. Sci. Res. 47, 1 (2010), 156--163.","journal-title":"Eur. J. Sci. Res."},{"key":"e_1_2_1_18_1","doi-asserted-by":"crossref","unstructured":"Eric Gaussier and Cyril Goutte. 2005. A probabilistic interpretation of precision recall and F-score with implication for evaluation. In Lect. Notes Comput. Sci. 3408. 345--359.  Eric Gaussier and Cyril Goutte. 2005. A probabilistic interpretation of precision recall and F-score with implication for evaluation. In Lect. Notes Comput. Sci. 3408. 345--359.","DOI":"10.1007\/978-3-540-31865-1_25"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/68814.68817"},{"key":"e_1_2_1_20_1","volume-title":"Proceedings of the 25th Workshop of the Psychology of Programming Interest Group.","author":"Hosseini Roya","year":"2014","unstructured":"Roya Hosseini , Arto Vihavainen , and Peter Brusilovsky . 2014 . Exploring problem solving paths in a java programming course . In Proceedings of the 25th Workshop of the Psychology of Programming Interest Group. Roya Hosseini, Arto Vihavainen, and Peter Brusilovsky. 2014. Exploring problem solving paths in a java programming course. In Proceedings of the 25th Workshop of the Psychology of Programming Interest Group."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858796.2858798"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1151588.1151600"},{"key":"e_1_2_1_23_1","unstructured":"Zlatko J. Kova\u010di\u0107 and J. S. Green. 2010. Predictive working tool for early identification of \u201cat risk\u201d students (2010).  Zlatko J. Kova\u010di\u0107 and J. S. Green. 2010. Predictive working tool for early identification of \u201cat risk\u201d students (2010)."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.5121\/ijdkp.2011.1504"},{"key":"e_1_2_1_25_1","volume-title":"Proceedings of the 19th Annual Workshop of the Psychology of Programming Interest Group, Joensuu, Finnland. Citeseer, 32--41","author":"Lahtinen Essi","year":"2007","unstructured":"Essi Lahtinen . 2007 . A categorization of novice programmers: A cluster analysis study . In Proceedings of the 19th Annual Workshop of the Psychology of Programming Interest Group, Joensuu, Finnland. Citeseer, 32--41 . Essi Lahtinen. 2007. A categorization of novice programmers: A cluster analysis study. In Proceedings of the 19th Annual Workshop of the Psychology of Programming Interest Group, Joensuu, Finnland. Citeseer, 32--41."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/800066.801357"},{"key":"e_1_2_1_27_1","unstructured":"Athanasios Papoulis. 1990. Probability and Statistics. Prentence-Hall International Editions.  Athanasios Papoulis. 1990. Probability and Statistics. Prentence-Hall International Editions."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2157136.2157182"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2632320.2632354"},{"key":"e_1_2_1_30_1","article-title":"Drop out feature of student data for academic performance using decision tree techniques","volume":"10","author":"Quadri M. N.","year":"2010","unstructured":"M. N. Quadri and N. V. Kalyankar . 2010 . Drop out feature of student data for academic performance using decision tree techniques . Global J. Comput. Sci. Technol. 10 , 2 (2010). M. N. Quadri and N. V. Kalyankar. 2010. Drop out feature of student data for academic performance using decision tree techniques. Global J. Comput. Sci. Technol. 10, 2 (2010).","journal-title":"Global J. Comput. Sci. Technol."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2960310.2960333"},{"key":"e_1_2_1_32_1","volume-title":"Fletcher","author":"Fletcher Suzanne W.","year":"2005","unstructured":"Suzanne W. Fletcher and Robert H . Fletcher . 2005 . Clinical Epidemiology : The Essentials. Vol. 1 . Lippincott Williams and Wilkins . Suzanne W. Fletcher and Robert H. Fletcher. 2005. Clinical Epidemiology: The Essentials. Vol. 1. Lippincott Williams and Wilkins."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1595496.1562929"},{"key":"e_1_2_1_34_1","first-page":"177","article-title":"Analyzing online protocols to characterize novice Java programmers","volume":"138","author":"Rodrigo Maria Mercedes T.","year":"2009","unstructured":"Maria Mercedes T. Rodrigo , Emily Tabanao , Maria Beatriz E. Lahoz , and Matthew C. Jadud . 2009 . Analyzing online protocols to characterize novice Java programmers . Philippine J. Sci. 138 , 2 (2009), 177 -- 190 . Maria Mercedes T. Rodrigo, Emily Tabanao, Maria Beatriz E. Lahoz, and Matthew C. Jadud. 2009. Analyzing online protocols to characterize novice Java programmers. Philippine J. Sci. 138, 2 (2009), 177--190.","journal-title":"Philippine J. Sci."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/1044550.1041669"},{"key":"e_1_2_1_36_1","first-page":"28","article-title":"Mathematical preparation as a basis for success in CS-II","volume":"17","author":"Stein Michael V.","year":"2002","unstructured":"Michael V. Stein . 2002 . Mathematical preparation as a basis for success in CS-II . J. Comput. Sci. Colleges 17 , 4 (2002), 28 -- 38 . Michael V. Stein. 2002. Mathematical preparation as a basis for success in CS-II. J. Comput. Sci. Colleges 17, 4 (2002), 28--38.","journal-title":"J. Comput. Sci. Colleges"},{"key":"e_1_2_1_37_1","volume-title":"Proceedings of the 14th Workshop of the Psychology of Programming Interest Group. 45--57","author":"Tukiainen Markku","year":"2002","unstructured":"Markku Tukiainen and Eero M\u00f6nkk\u00f6nen . 2002 . Programming aptitude testing as a prediction of learning to program . In Proceedings of the 14th Workshop of the Psychology of Programming Interest Group. 45--57 . Markku Tukiainen and Eero M\u00f6nkk\u00f6nen. 2002. Programming aptitude testing as a prediction of learning to program. In Proceedings of the 14th Workshop of the Psychology of Programming Interest Group. 45--57."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1080\/08993400500224419"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICALT.2013.161"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2462476.2462501"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICALT.2013.99"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2538862.2538930"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/5600.5701"},{"key":"e_1_2_1_44_1","volume-title":"Proceedings of the 16th Annual Workshop of the Psychology of Programming Interest Group. 97--109","author":"Wiedenbeck Susan","unstructured":"Susan Wiedenbeck , Deborah Labelle , and Vennila N. R. Kain . 2004. Factors affecting course outcomes in introductory programming . In Proceedings of the 16th Annual Workshop of the Psychology of Programming Interest Group. 97--109 . Susan Wiedenbeck, Deborah Labelle, and Vennila N. R. Kain. 2004. Factors affecting course outcomes in introductory programming. In Proceedings of the 16th Annual Workshop of the Psychology of Programming Interest Group. 97--109."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.2307\/2983604"},{"key":"e_1_2_1_46_1","volume-title":"Proceedings of the 7th International Conference on Educational Data Mining","author":"Yudelson Michael","year":"2014","unstructured":"Michael Yudelson , Roya Hosseini , Arto Vihavainen , and Peter Brusilovsky . 2014 . Investigating automated student modeling in a Java MOOC . In Proceedings of the 7th International Conference on Educational Data Mining 2014. Michael Yudelson, Roya Hosseini, Arto Vihavainen, and Peter Brusilovsky. 2014. Investigating automated student modeling in a Java MOOC. In Proceedings of the 7th International Conference on Educational Data Mining 2014."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2538862.2538878"}],"container-title":["ACM Transactions on Computing Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3123814","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3123814","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T19:05:27Z","timestamp":1750273527000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3123814"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,8,28]]},"references-count":47,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2017,9,30]]}},"alternative-id":["10.1145\/3123814"],"URL":"https:\/\/doi.org\/10.1145\/3123814","relation":{},"ISSN":["1946-6226"],"issn-type":[{"value":"1946-6226","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,8,28]]},"assertion":[{"value":"2016-09-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-05-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-08-28","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}