{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,16]],"date-time":"2026-03-16T20:37:54Z","timestamp":1773693474777,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":30,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,2,22]],"date-time":"2022-02-22T00:00:00Z","timestamp":1645488000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["DUE-1915257"],"award-info":[{"award-number":["DUE-1915257"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,2,22]]},"DOI":"10.1145\/3478431.3499388","type":"proceedings-article","created":{"date-parts":[[2022,2,23]],"date-time":"2022-02-23T00:53:28Z","timestamp":1645577608000},"page":"647-653","source":"Crossref","is-referenced-by-count":9,"title":["Are We Fair?"],"prefix":"10.1145","author":[{"given":"Max","family":"Fowler","sequence":"first","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}]},{"suffix":"IV","given":"David H.","family":"Smith","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}]},{"given":"Chinedu","family":"Emeka","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}]},{"given":"Matthew","family":"West","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}]},{"given":"Craig","family":"Zilles","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}]}],"member":"320","published-online":{"date-parts":[[2022,2,22]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1080\/08993400500150747"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.5220\/0006795104740480"},{"key":"e_1_3_2_1_3_1","unstructured":"Yigal Attali. 2018. Automatic Item Generation Unleashed: An Evaluation of a Large-Scale Deployment of Item Models. In Artificial Intelligence in Education Carolyn Penstein Ros\u00e9 Roberto Mart\u00ednez-Maldonado H. Ulrich Hoppe Rose Luckin Manolis Mavrikis Kaska Porayska-Pomsta Bruce McLaren and Benedict du Boulay (Eds.). Springer International Publishing Cham 17--29.  Yigal Attali. 2018. Automatic Item Generation Unleashed: An Evaluation of a Large-Scale Deployment of Item Models. In Artificial Intelligence in Education Carolyn Penstein Ros\u00e9 Roberto Mart\u00ednez-Maldonado H. Ulrich Hoppe Rose Luckin Manolis Mavrikis Kaska Porayska-Pomsta Bruce McLaren and Benedict du Boulay (Eds.). Springer International Publishing Cham 17--29."},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1187\/cbe.16-02-0104"},{"key":"e_1_3_2_1_6_1","first-page":"3","article-title":"Generation CS: The Mixed News on Diversity and the Enrollment Surge","volume":"8","author":"Camp Tracy","year":"2017","unstructured":"Tracy Camp , W. Richards Adrion , Betsy Bizot , Susan Davidson , Mary Hall , Susanne Hambrusch , Ellen Walker , and Stuart Zweben . 2017 . Generation CS: The Mixed News on Diversity and the Enrollment Surge . ACM Inroads 8 , 3 (July 2017), 36--42. https:\/\/doi.org\/10.1145\/3103175 Tracy Camp, W. Richards Adrion, Betsy Bizot, Susan Davidson, Mary Hall, Susanne Hambrusch, Ellen Walker, and Stuart Zweben. 2017. Generation CS: The Mixed News on Diversity and the Enrollment Surge. ACM Inroads 8, 3 (July 2017), 36--42. https:\/\/doi.org\/10.1145\/3103175","journal-title":"ACM Inroads"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3231644.3231664"},{"key":"e_1_3_2_1_8_1","volume-title":"Artificial Intelligence in Education, Seiji Isotani, Eva Mill\u00e1n","author":"Chen Binglin","unstructured":"Binglin Chen , Craig Zilles , Matthew West , and Timothy Bretl . 2019. Effect of Discrete and Continuous Parameter Variation on Difficulty in Automatic Item Generation . In Artificial Intelligence in Education, Seiji Isotani, Eva Mill\u00e1n , Amy Ogan, Peter Hastings, Bruce McLaren, and Rose Luckin (Eds.). Springer International Publishing , Cham , 71--83. Binglin Chen, Craig Zilles, Matthew West, and Timothy Bretl. 2019. Effect of Discrete and Continuous Parameter Variation on Difficulty in Automatic Item Generation. In Artificial Intelligence in Education, Seiji Isotani, Eva Mill\u00e1n, Amy Ogan, Peter Hastings, Bruce McLaren, and Rose Luckin (Eds.). Springer International Publishing, Cham, 71--83."},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEET.2019.00021"},{"key":"e_1_3_2_1_11_1","volume-title":"Investigating Item Bias in a CS1 Exam with Differential Item Functioning","author":"Davidson Matt J.","unstructured":"Matt J. Davidson , Brett Wortzman , Amy J. Ko , and Min Li. 2021. Investigating Item Bias in a CS1 Exam with Differential Item Functioning . Association for Computing Machinery , New York, NY, USA , 1142--1148. https:\/\/doi.org\/10.1145\/3408877. 3432397 Matt J. Davidson, Brett Wortzman, Amy J. Ko, and Min Li. 2021. Investigating Item Bias in a CS1 Exam with Differential Item Functioning. Association for Computing Machinery, New York, NY, USA, 1142--1148. https:\/\/doi.org\/10.1145\/3408877. 3432397"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.3758\/s13428-017-1004-5"},{"key":"e_1_3_2_1_13_1","volume-title":"Proceedings of the 13th Annual Conference on Innovation and Technology in Computer Science Education","author":"Stephen","unstructured":"Stephen H. Edwards and Manuel A. Perez-Quinones. 2008. Web-CAT: Automatically Grading Programming Assignments . In Proceedings of the 13th Annual Conference on Innovation and Technology in Computer Science Education ( Madrid, Spain) (ITiCSE '08). Association for Computing Machinery, New York, NY, USA, 328. https:\/\/doi.org\/10.1145\/1384271.1384371 Stephen H. Edwards and Manuel A. Perez-Quinones. 2008. Web-CAT: Automatically Grading Programming Assignments. In Proceedings of the 13th Annual Conference on Innovation and Technology in Computer Science Education (Madrid, Spain) (ITiCSE '08). Association for Computing Machinery, New York, NY, USA, 328. https:\/\/doi.org\/10.1145\/1384271.1384371"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3372782.3406275"},{"key":"e_1_3_2_1_15_1","volume-title":"Alternative to Proctoring in Introductory Statistics Community College Courses. Walden Dissertations and Doctoral Studies (Jan","author":"Feinman Lena","year":"2018","unstructured":"Lena Feinman . 2018. Alternative to Proctoring in Introductory Statistics Community College Courses. Walden Dissertations and Doctoral Studies (Jan . 2018 ). https:\/\/scholarworks.waldenu.edu\/dissertations\/4622 Lena Feinman. 2018. Alternative to Proctoring in Introductory Statistics Community College Courses. Walden Dissertations and Doctoral Studies (Jan. 2018). https:\/\/scholarworks.waldenu.edu\/dissertations\/4622"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1080\/10511970.2020.1772915"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287324.3287354"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3408877.3432413"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"crossref","unstructured":"M.J Gierl and T.M Haladyna (Eds.). 2012. Automatic Item Generation: Theory and Practice (1 ed.). Routledge. https:\/\/doi.org\/10.4324\/9780203803912  M.J Gierl and T.M Haladyna (Eds.). 2012. Automatic Item Generation: Theory and Practice (1 ed.). Routledge. https:\/\/doi.org\/10.4324\/9780203803912","DOI":"10.4324\/9780203803912"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2007.10.006"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1002\/acp.3605"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1186\/s41239-020-00227-w"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1080\/08832323.2017.1417231"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1080\/1097198X.2018.1542262"},{"key":"e_1_3_2_1_25_1","volume-title":"Reducing Difficulty Variance in Randomized Assessments. In 2019 ASEE Annual Conference & Exposition. ASEE Conferences","author":"Paras Sud Matthew West","year":"2019","unstructured":"Matthew West Paras Sud and Craig Zilles . 2019 . Reducing Difficulty Variance in Randomized Assessments. In 2019 ASEE Annual Conference & Exposition. ASEE Conferences , Tampa, Florida. https:\/\/doi.org\/10. 18260\/1--2--33228 Matthew West Paras Sud and Craig Zilles. 2019. Reducing Difficulty Variance in Randomized Assessments. In 2019 ASEE Annual Conference & Exposition. ASEE Conferences, Tampa, Florida. https:\/\/doi.org\/10.18260\/1--2--33228"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3408877.3432556"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2839509.2844621"},{"key":"e_1_3_2_1_28_1","first-page":"439","article-title":"Challenges of online exam, performances and problems for online university exam","volume":"10","author":"Sarrayrih Mohammad A","year":"2013","unstructured":"Mohammad A Sarrayrih and Mohammed Ilyas . 2013 . Challenges of online exam, performances and problems for online university exam . International Journal of Computer Science Issues (IJCSI) 10 , 1 (2013), 439 . Mohammad A Sarrayrih and Mohammed Ilyas. 2013. Challenges of online exam, performances and problems for online university exam. International Journal of Computer Science Issues (IJCSI) 10, 1 (2013), 439.","journal-title":"International Journal of Computer Science Issues (IJCSI)"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevSTPER.2.020102"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3328778.3366859"},{"key":"e_1_3_2_1_31_1","article-title":"Student Perceptions of Cheating in Online Business Courses","author":"Watters Michael P.","year":"2011","unstructured":"Michael P. Watters , Paul J. Robertson , and Renae K. Clark . 2011 . Student Perceptions of Cheating in Online Business Courses . Journal of Instructional Pedagogies 6 ( Sept. 2011). https:\/\/eric.ed.gov\/?id=EJ1097041 Michael P. Watters, Paul J. Robertson, and Renae K. Clark. 2011. Student Perceptions of Cheating in Online Business Courses. Journal of Instructional Pedagogies 6 (Sept. 2011). https:\/\/eric.ed.gov\/?id=EJ1097041","journal-title":"Journal of Instructional Pedagogies 6"},{"key":"e_1_3_2_1_32_1","unstructured":"Matthew West Geoffrey L. Herman and Craig Zilles. 2015. PrairieLearn: Mastery-based Online Problem Solving with Adaptive Scoring and Recommendations Driven by Machine Learning. 26.1238.1--26.1238.14. https:\/\/peer.asee.org\/prairielearn-mastery-based-online-problem-solvingwith-adaptive-scoring-and-recommendations-driven-by-machine-learning  Matthew West Geoffrey L. Herman and Craig Zilles. 2015. PrairieLearn: Mastery-based Online Problem Solving with Adaptive Scoring and Recommendations Driven by Machine Learning. 26.1238.1--26.1238.14. https:\/\/peer.asee.org\/prairielearn-mastery-based-online-problem-solvingwith-adaptive-scoring-and-recommendations-driven-by-machine-learning"}],"event":{"name":"SIGCSE 2022: The 53rd ACM Technical Symposium on Computer Science Education","location":"Providence RI USA","acronym":"SIGCSE 2022","sponsor":["SIGCSE ACM Special Interest Group on Computer Science Education"]},"container-title":["Proceedings of the 53rd ACM Technical Symposium on Computer Science Education"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3478431.3499388","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3478431.3499388","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3478431.3499388","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:31:14Z","timestamp":1750188674000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3478431.3499388"}},"subtitle":["Quantifying Score Impacts of Computer Science Exams with Randomized Question Pools"],"short-title":[],"issued":{"date-parts":[[2022,2,22]]},"references-count":30,"alternative-id":["10.1145\/3478431.3499388","10.1145\/3478431"],"URL":"https:\/\/doi.org\/10.1145\/3478431.3499388","relation":{},"subject":[],"published":{"date-parts":[[2022,2,22]]}}}