{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,26]],"date-time":"2026-06-26T23:38:28Z","timestamp":1782517108830,"version":"3.54.5"},"publisher-location":"New York, NY, USA","reference-count":47,"publisher":"ACM","license":[{"start":{"date-parts":[[2025,3,3]],"date-time":"2025-03-03T00:00:00Z","timestamp":1740960000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,3,3]]},"DOI":"10.1145\/3706468.3706540","type":"proceedings-article","created":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T14:04:11Z","timestamp":1740146651000},"page":"547-557","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Bias or Insufficient Sample Size? Improving Reliable Estimation of Algorithmic Bias for Minority Groups"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8893-7898","authenticated-orcid":false,"given":"Jaeyoon","family":"Choi","sequence":"first","affiliation":[{"name":"University of California, Irvine, Irvine, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7920-4510","authenticated-orcid":false,"given":"Shamya","family":"Karumbaiah","sequence":"additional","affiliation":[{"name":"University of Wisconsin-Madison, Madison, Wisconsin, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1321-8159","authenticated-orcid":false,"given":"Jeffrey","family":"Matayoshi","sequence":"additional","affiliation":[{"name":"McGraw Hill ALEKS, Irvine, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,3,3]]},"reference":[{"key":"e_1_3_3_2_2_2","doi-asserted-by":"crossref","unstructured":"Su\u00a0Lin Blodgett Solon Barocas Hal Daum\u00e9\u00a0III and Hanna Wallach. 2020. Language (technology) is power: A critical survey of\" bias\" in nlp. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2005.14050 (2020).","DOI":"10.18653\/v1\/2020.acl-main.485"},{"key":"e_1_3_3_2_3_2","doi-asserted-by":"crossref","unstructured":"Allan\u00a0M Brandt. 1978. Racism and research: the case of the Tuskegee Syphilis Study. Hastings center report (1978) 21\u201329.","DOI":"10.2307\/3561468"},{"key":"e_1_3_3_2_4_2","doi-asserted-by":"crossref","unstructured":"Lawrence Brown and Xuefeng Li. 2005. Confidence intervals for two sample binomial distribution. Journal of Statistical Planning and Inference 130 1-2 (2005) 359\u2013375.","DOI":"10.1016\/j.jspi.2003.09.039"},{"key":"e_1_3_3_2_5_2","first-page":"77","volume-title":"Conference on fairness, accountability and transparency","author":"Buolamwini Joy","year":"2018","unstructured":"Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency. PMLR, 77\u201391."},{"key":"e_1_3_3_2_6_2","doi-asserted-by":"crossref","unstructured":"Katherine\u00a0S Button John\u00a0PA Ioannidis Claire Mokrysz Brian\u00a0A Nosek Jonathan Flint Emma\u00a0SJ Robinson and Marcus\u00a0R Munaf\u00f2. 2013. Power failure: why small sample size undermines the reliability of neuroscience. Nature reviews neuroscience 14 5 (2013) 365\u2013376.","DOI":"10.1038\/nrn3475"},{"key":"e_1_3_3_2_7_2","doi-asserted-by":"crossref","unstructured":"Susan Carey. 2004. Bootstrapping & the origin of concepts. Daedalus 133 1 (2004) 59\u201368.","DOI":"10.1162\/001152604772746701"},{"key":"e_1_3_3_2_8_2","doi-asserted-by":"crossref","unstructured":"Su Chen Ying Fang Genghu Shi John Sabatini Daphne Greenberg Jan Frijters and Arthur\u00a0C Graesser. 2021. Automated disengagement tracking within an intelligent tutoring system. Frontiers in artificial intelligence 3 (2021) 595627.","DOI":"10.3389\/frai.2020.595627"},{"key":"e_1_3_3_2_9_2","doi-asserted-by":"crossref","unstructured":"Davide Chicco and Giuseppe Jurman. 2020. The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC genomics 21 (2020) 1\u201313.","DOI":"10.1186\/s12864-019-6413-7"},{"key":"e_1_3_3_2_10_2","doi-asserted-by":"crossref","unstructured":"Alexandra Chouldechova. 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data 5 2 (2017) 153\u2013163.","DOI":"10.1089\/big.2016.0047"},{"key":"e_1_3_3_2_11_2","doi-asserted-by":"crossref","unstructured":"Alexandra Chouldechova and Aaron Roth. 2020. A snapshot of the frontiers of fairness in machine learning. Commun. ACM 63 5 (2020) 82\u201389.","DOI":"10.1145\/3376898"},{"key":"e_1_3_3_2_12_2","doi-asserted-by":"crossref","unstructured":"Natasha Codiroli\u00a0Mcmaster and Rose Cook. 2019. The contribution of intersectionality to quantitative research into educational inequalities. Review of Education 7 2 (2019) 271\u2013292.","DOI":"10.1002\/rev3.3116"},{"key":"e_1_3_3_2_13_2","unstructured":"Damien Dablain Bartosz Krawczyk and Nitesh Chawla. 2022. Towards a holistic view of bias in machine learning: Bridging algorithmic fairness and imbalanced learning. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2207.06084 (2022)."},{"key":"e_1_3_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/3027385.3027405"},{"key":"e_1_3_3_2_15_2","doi-asserted-by":"crossref","unstructured":"Thomas\u00a0J Diciccio and Joseph\u00a0P Romano. 1988. A review of bootstrap confidence intervals. Journal of the Royal Statistical Society Series B: Statistical Methodology 50 3 (1988) 338\u2013354.","DOI":"10.1111\/j.2517-6161.1988.tb01732.x"},{"key":"e_1_3_3_2_16_2","doi-asserted-by":"crossref","unstructured":"Morten\u00a0W Fagerland Stian Lydersen and Petter Laake. 2015. Recommended confidence intervals for two independent binomial proportions. Statistical methods in medical research 24 2 (2015) 224\u2013254.","DOI":"10.1177\/0962280211415469"},{"key":"e_1_3_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/2783258.2783311"},{"key":"e_1_3_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICTAI.2019.00131"},{"key":"e_1_3_3_2_19_2","unstructured":"Moritz Hardt Eric Price and Nati Srebro. 2016. Equality of opportunity in supervised learning. Advances in neural information processing systems 29 (2016)."},{"key":"e_1_3_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACII.2013.47"},{"key":"e_1_3_3_2_21_2","unstructured":"Haewon Jeong Michael\u00a0D Wu Nilanjana Dasgupta Muriel M\u00e9dard and Flavio Calmon. 2022. Who gets the benefit of the doubt? Racial bias in machine learning algorithms applied to secondary school math education. Math AI for Education: Bridging the Gap Between Research and Smart Education (2022)."},{"key":"e_1_3_3_2_22_2","unstructured":"Shamya Karumbaiah Ryan\u00a0S Baker and Valerie Shute. 2018. Predicting Quitting in Students Playing a Learning Game. International Educational Data Mining Society (2018)."},{"key":"e_1_3_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/RESPECT51740.2021.9620605"},{"key":"e_1_3_3_2_24_2","doi-asserted-by":"crossref","unstructured":"Shamya Karumbaiah Jaclyn Ocumpaugh and Ryan\u00a0S Baker. 2022. Context matters: Differing implications of motivation and help-seeking in educational technology. International Journal of Artificial Intelligence in Education 32 3 (2022) 685\u2013724.","DOI":"10.1007\/s40593-021-00272-0"},{"key":"e_1_3_3_2_25_2","first-page":"2564","volume-title":"International conference on machine learning","author":"Kearns Michael","year":"2018","unstructured":"Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei\u00a0Steven Wu. 2018. Preventing fairness gerrymandering: Auditing and learning for subgroup fairness. In International conference on machine learning. PMLR, 2564\u20132572."},{"key":"e_1_3_3_2_26_2","doi-asserted-by":"crossref","unstructured":"Gregory\u00a0A Kimble. 1987. The scientific value of undergraduate research participation. American Psychological Association (1987).","DOI":"10.1037\/\/0003-066X.42.3.267.b"},{"key":"e_1_3_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3593013.3594100"},{"key":"e_1_3_3_2_28_2","doi-asserted-by":"crossref","unstructured":"Lifeng Lin. 2018. Bias caused by sampling error in meta-analysis with small sample sizes. PloS one 13 9 (2018) e0204056.","DOI":"10.1371\/journal.pone.0204056"},{"key":"e_1_3_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-72657-7_16"},{"key":"e_1_3_3_2_30_2","doi-asserted-by":"crossref","unstructured":"Ninareh Mehrabi Fred Morstatter Nripsuta Saxena Kristina Lerman and Aram Galstyan. 2021. A survey on bias and fairness in machine learning. ACM computing surveys (CSUR) 54 6 (2021) 1\u201335.","DOI":"10.1145\/3457607"},{"key":"e_1_3_3_2_31_2","doi-asserted-by":"crossref","unstructured":"Shira Mitchell Eric Potash Solon Barocas Alexander D\u2019Amour and Kristian Lum. 2021. Algorithmic fairness: Choices assumptions and definitions. Annual review of statistics and its application 8 1 (2021) 141\u2013163.","DOI":"10.1146\/annurev-statistics-042720-125902"},{"key":"e_1_3_3_2_32_2","doi-asserted-by":"crossref","unstructured":"Robert\u00a0G Newcombe. 1998. Two-sided confidence intervals for the single proportion: comparison of seven methods. Statistics in medicine 17 8 (1998) 857\u2013872.","DOI":"10.1002\/(SICI)1097-0258(19980430)17:8<857::AID-SIM777>3.0.CO;2-E"},{"key":"e_1_3_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.4324\/9781003141464-25"},{"key":"e_1_3_3_2_34_2","unstructured":"Jennifer\u00a0K Olsen Vincent Aleven and Nikol Rummel. 2015. Predicting Student Performance in a Collaborative Learning Environment. International Educational Data Mining Society (2015)."},{"key":"e_1_3_3_2_35_2","doi-asserted-by":"crossref","unstructured":"Sama Ranjeeth Thamarai\u00a0Pugazhendhi Latchoumi and P\u00a0Victer Paul. 2020. A survey on predictive models of learning analytics. Procedia Computer Science 167 (2020) 37\u201346.","DOI":"10.1016\/j.procs.2020.03.180"},{"key":"e_1_3_3_2_36_2","unstructured":"Pedro Saleiro Benedict Kuester Loren Hinkson Jesse London Abby Stevens Ari Anisfeld Kit\u00a0T Rodolfa and Rayid Ghani. 2018. Aequitas: A bias and fairness audit toolkit. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/1811.05577 (2018)."},{"key":"e_1_3_3_2_37_2","doi-asserted-by":"crossref","unstructured":"Salome\u00a0E Scholtz. 2021. Sacrifice is a step beyond convenience: A review of convenience sampling in psychological research in Africa. SA Journal of Industrial Psychology 47 1 (2021) 1\u201312.","DOI":"10.4102\/sajip.v47i0.1837"},{"key":"e_1_3_3_2_38_2","doi-asserted-by":"crossref","unstructured":"Philip Sedgwick. 2012. What is sampling error? Bmj 344 (2012).","DOI":"10.1136\/bmj.e4285"},{"key":"e_1_3_3_2_39_2","doi-asserted-by":"crossref","unstructured":"Lele Sha Mladen Rakovi\u0107 Angel Das Dragan Ga\u0161evi\u0107 and Guanliang Chen. 2022. Leveraging class balancing techniques to alleviate algorithmic bias for predictive tasks in education. IEEE Transactions on Learning Technologies 15 4 (2022) 481\u2013492.","DOI":"10.1109\/TLT.2022.3196278"},{"key":"e_1_3_3_2_40_2","unstructured":"Ajay\u00a0S Singh and Micah\u00a0B Masuku. 2014. Sampling techniques & determination of sample size in applied statistics research: An overview. International Journal of economics commerce and management 2 11 (2014) 1\u201322."},{"key":"e_1_3_3_2_41_2","doi-asserted-by":"crossref","unstructured":"Helen Smith. 2020. Algorithmic bias: should students pay the price? AI & society 35 4 (2020) 1077\u20131078.","DOI":"10.1007\/s00146-020-01054-3"},{"key":"e_1_3_3_2_42_2","doi-asserted-by":"crossref","unstructured":"Linda\u00a0J Smith. 2008. How ethical is ethical research? Recruiting marginalized vulnerable groups into health services research. Journal of Advanced nursing 62 2 (2008) 248\u2013257.","DOI":"10.1111\/j.1365-2648.2007.04567.x"},{"key":"e_1_3_3_2_43_2","doi-asserted-by":"crossref","unstructured":"KP Suresh and Sachin Chandrashekara. 2012. Sample size estimation and power analysis for clinical research studies. Journal of human reproductive sciences 5 1 (2012) 7\u201313.","DOI":"10.4103\/0974-1208.97779"},{"key":"e_1_3_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/2460296.2460337"},{"key":"e_1_3_3_2_45_2","doi-asserted-by":"crossref","unstructured":"Ga\u00ebl Varoquaux. 2018. Cross-validation failure: Small sample sizes lead to large error bars. Neuroimage 180 (2018) 68\u201377.","DOI":"10.1016\/j.neuroimage.2017.06.061"},{"key":"e_1_3_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3636555.3636890"},{"key":"e_1_3_3_2_47_2","unstructured":"Jiayi Zhang Juliana Ma Alexandra\u00a0L Andres Stephen Hutt Ryan\u00a0S Baker Jaclyn Ocumpaugh Nidhi Nasiar Caitlin Mills Jamiella Brooks Sheela Sethuaman Tyron Young et\u00a0al. 2022. Using machine learning to detect SMART model cognitive operations in mathematical problem-solving process. Journal of Educational Data Mining 14 3 (2022) 76\u2013108."},{"key":"e_1_3_3_2_48_2","doi-asserted-by":"crossref","unstructured":"Amin Zollanvari Refik\u00a0Caglar Kizilirmak Yau\u00a0Hee Kho and Daniel Hern\u00e1ndez-Torrano. 2017. Predicting students\u2019 GPA and developing intervention strategies based on self-regulatory learning behaviors. IEEE Access 5 (2017) 23792\u201323802.","DOI":"10.1109\/ACCESS.2017.2740980"}],"event":{"name":"LAK '25: The 15th International Learning Analytics and Knowledge Conference","location":"Dublin Ireland","acronym":"LAK 2025"},"container-title":["Proceedings of the 15th International Learning Analytics and Knowledge Conference"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3706468.3706540","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3706468.3706540","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:56:51Z","timestamp":1750298211000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3706468.3706540"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,3]]},"references-count":47,"alternative-id":["10.1145\/3706468.3706540","10.1145\/3706468"],"URL":"https:\/\/doi.org\/10.1145\/3706468.3706540","relation":{},"subject":[],"published":{"date-parts":[[2025,3,3]]},"assertion":[{"value":"2025-03-03","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}