{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,4]],"date-time":"2026-05-04T10:27:31Z","timestamp":1777890451558,"version":"3.51.4"},"reference-count":55,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2019,1,16]],"date-time":"2019-01-16T00:00:00Z","timestamp":1547596800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"NSF","doi-asserted-by":"publisher","award":["1712508"],"award-info":[{"award-number":["1712508"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Educ."],"published-print":{"date-parts":[[2019,9,30]]},"abstract":"<jats:p>As enrollments and class sizes in postsecondary institutions have increased, instructors have sought automated and lightweight means to identify students who are at risk of performing poorly in a course. This identification must be performed early enough in the term to allow instructors to assist those students before they fall irreparably behind. This study describes a modeling methodology that predicts student final exam scores in the third week of the term by using the clicker data that is automatically collected for instructors when they employ the Peer Instruction pedagogy. The modeling technique uses a support vector machine binary classifier, trained on one term of a course, to predict outcomes in the subsequent term. We applied this modeling technique to five different courses across the computer science curriculum, taught by three different instructors at two different institutions. Our modeling approach includes a set of strengths not seen wholesale in prior work, while maintaining competitive levels of accuracy with that work. These strengths include using a lightweight source of student data, affording early detection of struggling students, and predicting outcomes across terms in a natural setting (different final exams, minor changes to course content), across multiple courses in a curriculum, and across multiple institutions.<\/jats:p>","DOI":"10.1145\/3277569","type":"journal-article","created":{"date-parts":[[2019,1,16]],"date-time":"2019-01-16T13:13:32Z","timestamp":1547644412000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":74,"title":["A Robust Machine Learning Technique to Predict Low-performing Students"],"prefix":"10.1145","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7368-252X","authenticated-orcid":false,"given":"Soohyun Nam","family":"Liao","sequence":"first","affiliation":[{"name":"University of California, San Diego, USA"}]},{"given":"Daniel","family":"Zingaro","sequence":"additional","affiliation":[{"name":"University of Toronto Mississauga, Canada"}]},{"given":"Kevin","family":"Thai","sequence":"additional","affiliation":[{"name":"University of California, San Diego, USA"}]},{"given":"Christine","family":"Alvarado","sequence":"additional","affiliation":[{"name":"University of California, San Diego, USA"}]},{"given":"William G.","family":"Griswold","sequence":"additional","affiliation":[{"name":"University of California, San Diego, USA"}]},{"given":"Leo","family":"Porter","sequence":"additional","affiliation":[{"name":"University of California, San Diego, USA"}]}],"member":"320","published-online":{"date-parts":[[2019,1,16]]},"reference":[{"key":"e_1_2_1_1_1","first-page":"2","article-title":"Could final year school grades suffice as a predictor for future performance","volume":"15","author":"Abdulrazzaq Yousef Mohamed","year":"2009","unstructured":"Yousef Mohamed Abdulrazzaq and Khalil Ibrahim Qayed . 2009 . Could final year school grades suffice as a predictor for future performance ? Med. Teach. 15 , 2 -- 3 (2009), 243--251. Yousef Mohamed Abdulrazzaq and Khalil Ibrahim Qayed. 2009. Could final year school grades suffice as a predictor for future performance? Med. Teach. 15, 2--3 (2009), 243--251.","journal-title":"Med. Teach."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3123814"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2493394.2493416"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2787622.2787717"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.84.2.191"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1080\/08993400600997096"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3120259"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2787622.2787710"},{"key":"e_1_2_1_9_1","unstructured":"Jennifer M. Case. 2015. A different route to reducing university drop-out rates. In The Conversation. Retrieved from https:\/\/theconversation.com\/a-different-route-to-reducing-university-drop-out-rates-40406.  Jennifer M. Case. 2015. A different route to reducing university drop-out rates. In The Conversation. Retrieved from https:\/\/theconversation.com\/a-different-route-to-reducing-university-drop-out-rates-40406."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3017680.3017792"},{"key":"e_1_2_1_11_1","volume-title":"Proceedings of the Workshops held at Educational Data Mining: Workshop Approaching Twenty Years of Knowledge Tracing.","author":"Cengiz Nihat","year":"2014","unstructured":"Nihat Cengiz and Arban Uka . 2014 . Prediction of student success using enrollment data . Proceedings of the Workshops held at Educational Data Mining: Workshop Approaching Twenty Years of Knowledge Tracing. Nihat Cengiz and Arban Uka. 2014. Prediction of student success using enrollment data. Proceedings of the Workshops held at Educational Data Mining: Workshop Approaching Twenty Years of Knowledge Tracing."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF01099821"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1022627411411"},{"key":"e_1_2_1_14_1","volume-title":"Generation CS: Computer science undergraduate enrollments surge since","author":"CRA Enrollment Committee Institution Subgroup.","year":"2017","unstructured":"CRA Enrollment Committee Institution Subgroup. 2017 . Generation CS: Computer science undergraduate enrollments surge since 2006. Computing Research Association . Retrieved from http:\/\/cra.org\/data\/Generation-CS\/. CRA Enrollment Committee Institution Subgroup. 2017. Generation CS: Computer science undergraduate enrollments surge since 2006. Computing Research Association. Retrieved from http:\/\/cra.org\/data\/Generation-CS\/."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1119\/1.1374249"},{"key":"e_1_2_1_16_1","first-page":"407","article-title":"Approaches to learning in computer programming students and their effect on success","volume":"28","author":"de Raadt Michael","year":"2005","unstructured":"Michael de Raadt , Margaret Hamilton , Raymond Lister , Jodi Tutty , Bob Baker , Ilona Box , Quintin Cutts , Sally Fincher , John Hamer , Patricia Haden , Marian Petre , Anthony Robins , Simon, Ken Sutton , and Denise Tolhurst . 2005 . Approaches to learning in computer programming students and their effect on success . Res. Dev. Higher Educ.: Higher Educ. Chang. World 28 (2005), 407 -- 414 . Michael de Raadt, Margaret Hamilton, Raymond Lister, Jodi Tutty, Bob Baker, Ilona Box, Quintin Cutts, Sally Fincher, John Hamer, Patricia Haden, Marian Petre, Anthony Robins, Simon, Ken Sutton, and Denise Tolhurst. 2005. Approaches to learning in computer programming students and their effect on success. Res. Dev. Higher Educ.: Higher Educ. Chang. World 28 (2005), 407--414.","journal-title":"Res. Dev. Higher Educ.: Higher Educ. Chang. World"},{"key":"e_1_2_1_17_1","volume-title":"Lindsay","author":"Elias Edward M.","year":"1968","unstructured":"Edward M. Elias and Carl A . Lindsay . 1968 . The Role of Intellective Variables in Achievement and Attrition of Associate Degree Students at the York Campus for the Years 1959 to 1963. Technical Report PSU -68 -7. Pennsylvania State University . Edward M. Elias and Carl A. Lindsay. 1968. The Role of Intellective Variables in Achievement and Attrition of Associate Degree Students at the York Campus for the Years 1959 to 1963. Technical Report PSU -68 -7. Pennsylvania State University."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1148\/radiology.143.1.7063747"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.4018\/javet.2011100101"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1151588.1151600"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1046\/j.1365-2923.2001.01042.x"},{"key":"e_1_2_1_22_1","unstructured":"Cynthia Lee Leo Porter Beth Simon and Daniel Zingaro. 2012. Peer instruction for computer science. Retrieved from http:\/\/www.peerinstruction4cs.org.  Cynthia Lee Leo Porter Beth Simon and Daniel Zingaro. 2012. Peer instruction for computer science. Retrieved from http:\/\/www.peerinstruction4cs.org."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2499947.2499949"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2960310.2960315"},{"key":"e_1_2_1_25_1","first-page":"219","article-title":"Using peer instruction and I-clickers to enhance student participation in calculus","volume":"19","author":"Lucas Adam","year":"2009","unstructured":"Adam Lucas . 2009 . Using peer instruction and I-clickers to enhance student participation in calculus . Prob. Resour. Issues Math. Undergrad. Studies 19 , 3 (2009), 219 -- 231 . Adam Lucas. 2009. Using peer instruction and I-clickers to enhance student participation in calculus. Prob. Resour. Issues Math. Undergrad. Studies 19, 3 (2009), 219--231.","journal-title":"Prob. Resour. Issues Math. Undergrad. Studies"},{"key":"e_1_2_1_26_1","volume-title":"Total undergraduate fall enrollment in degree-granting postsecondary institutions, by attendance status, sex of student, and control and level of institution: Selected years","author":"National Center for Education Statistics. 2016.","year":"1970","unstructured":"National Center for Education Statistics. 2016. Total undergraduate fall enrollment in degree-granting postsecondary institutions, by attendance status, sex of student, and control and level of institution: Selected years , 1970 through 2026. National Center for Education Statistics . https:\/\/nces.ed.gov\/programs\/digest\/d16\/tables\/dt16_303.70.asp. National Center for Education Statistics. 2016. Total undergraduate fall enrollment in degree-granting postsecondary institutions, by attendance status, sex of student, and control and level of institution: Selected years, 1970 through 2026. National Center for Education Statistics. https:\/\/nces.ed.gov\/programs\/digest\/d16\/tables\/dt16_303.70.asp."},{"key":"e_1_2_1_27_1","first-page":"182","article-title":"Predicting academic success in introduction to computers","volume":"12","author":"Petersen Charles G.","year":"1979","unstructured":"Charles G. Petersen and Trevor G. Howe . 1979 . Predicting academic success in introduction to computers . Assoc. Educ. Data Syst. 12 , 4 (1979), 182 -- 191 . Charles G. Petersen and Trevor G. Howe. 1979. Predicting academic success in introduction to computers. Assoc. Educ. Data Syst. 12, 4 (1979), 182--191.","journal-title":"Assoc. Educ. Data Syst."},{"key":"e_1_2_1_28_1","first-page":"185","article-title":"Peer instruction in physics and mathematics","volume":"11","author":"Pilzer Scott","year":"2001","unstructured":"Scott Pilzer . 2001 . Peer instruction in physics and mathematics . Prob. Resour. Issues Math. Undergrad. Studies 11 , 2 (2001), 185 -- 192 . Scott Pilzer. 2001. Peer instruction in physics and mathematics. Prob. Resour. Issues Math. Undergrad. Studies 11, 2 (2001), 185--192.","journal-title":"Prob. Resour. Issues Math. Undergrad. Studies"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2445196.2445250"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1999747.1999788"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2839509.2844642"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2462476.2465587"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2445196.2445248"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2538862.2538912"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2632320.2632354"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/s13042-011-0012-5"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1080\/08993401003612167"},{"key":"e_1_2_1_38_1","unstructured":"John E. Roueche. 1967. Research studies of the junior college dropout. Amer. Assoc. Junior Coll. (1967) 1--5.  John E. Roueche. 1967. Research studies of the junior college dropout. Amer. Assoc. Junior Coll. (1967) 1--5."},{"key":"e_1_2_1_39_1","first-page":"1","article-title":"Advanced placement exam scores as a predictor of performance in introductory college biology, chemistry and physics courses. Sci","volume":"16","author":"Sadler Philip M.","year":"2007","unstructured":"Philip M. Sadler and Robert H. Tai . 2007 . Advanced placement exam scores as a predictor of performance in introductory college biology, chemistry and physics courses. Sci . Educat. 16 , 2 (2007), 1 -- 19 . Philip M. Sadler and Robert H. Tai. 2007. Advanced placement exam scores as a predictor of performance in introductory college biology, chemistry and physics courses. Sci. Educat. 16, 2 (2007), 1--19.","journal-title":"Educat."},{"key":"e_1_2_1_40_1","volume-title":"Cohen Sadler and Frederic L. Kockesen Levent","author":"William","year":"1997","unstructured":"William E. Cohen Sadler and Frederic L. Kockesen Levent . 1997 . Factors affecting retention behavior: A model to predict at-risk students. In Proceedings of the Association for Institutional Research Annual Forum . William E. Cohen Sadler and Frederic L. Kockesen Levent. 1997. Factors affecting retention behavior: A model to predict at-risk students. In Proceedings of the Association for Institutional Research Annual Forum."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1016\/0360-1315(86)90031-X"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.2147\/AMEP.S19391"},{"key":"e_1_2_1_43_1","unstructured":"Karedn Shakerdge. 2016. High failure rates spur universities to overhaul math class. In The Hechinger Report. Retrieved from http:\/\/hechingerreport.org\/high-failure-rates-spur-universities-overhaul-math-class\/.  Karedn Shakerdge. 2016. High failure rates spur universities to overhaul math class. In The Hechinger Report. Retrieved from http:\/\/hechingerreport.org\/high-failure-rates-spur-universities-overhaul-math-class\/."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1080\/15602214031000112221"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/1734263.1734381"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/2445196.2445215"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11162-010-9170-7"},{"key":"e_1_2_1_48_1","volume-title":"Why peer discussion improves student performance on in-class concept questions. Science 323, 5910","author":"Smith Michelle K.","year":"2009","unstructured":"Michelle K. Smith , William B. Wood , Wendy K. Adams , Carl E. Wieman , Jennifer K. Knight , Nancy Guild , and Tin Tin Su. 2009. Why peer discussion improves student performance on in-class concept questions. Science 323, 5910 ( 2009 ), 122--124. Michelle K. Smith, William B. Wood, Wendy K. Adams, Carl E. Wieman, Jennifer K. Knight, Nancy Guild, and Tin Tin Su. 2009. Why peer discussion improves student performance on in-class concept questions. Science 323, 5910 (2009), 122--124."},{"key":"e_1_2_1_49_1","volume-title":"Kernel-based machine-learning lab.","author":"Smola Alex","unstructured":"Alex Smola , Kurt Hornik , Achim Zeileis , and Alexandros Karatzoglou . 2003. Kernel-based machine-learning lab. Retrieved from https:\/\/www.rdocumentation.org\/packages\/kernlab\/versions\/0.9-25. Alex Smola, Kurt Hornik, Achim Zeileis, and Alexandros Karatzoglou. 2003. Kernel-based machine-learning lab. Retrieved from https:\/\/www.rdocumentation.org\/packages\/kernlab\/versions\/0.9-25."},{"key":"e_1_2_1_50_1","unstructured":"Louise Tickle. 2015. How universities are using data to stop students dropping out. In The Guardian. Retrieved from https:\/\/www.theguardian.com\/guardian-professional\/.  Louise Tickle. 2015. How universities are using data to stop students dropping out. In The Guardian. Retrieved from https:\/\/www.theguardian.com\/guardian-professional\/."},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIPRO.2014.6859754"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICALT.2013.99"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/364447.364581"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-02738-8_7"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevPhysEducRes.113.010104"}],"container-title":["ACM Transactions on Computing Education"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3277569","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3277569","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3277569","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T01:39:38Z","timestamp":1750210778000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3277569"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,1,16]]},"references-count":55,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2019,9,30]]}},"alternative-id":["10.1145\/3277569"],"URL":"https:\/\/doi.org\/10.1145\/3277569","relation":{},"ISSN":["1946-6226"],"issn-type":[{"value":"1946-6226","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,1,16]]},"assertion":[{"value":"2018-05-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-08-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-01-16","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}