{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T16:01:39Z","timestamp":1780329699784,"version":"3.54.1"},"publisher-location":"New York, NY, USA","reference-count":50,"publisher":"ACM","license":[{"start":{"date-parts":[[2026,6,7]],"date-time":"2026-06-07T00:00:00Z","timestamp":1780790400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2026,6,8]]},"DOI":"10.1145\/3774935.3806189","type":"proceedings-article","created":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T14:44:13Z","timestamp":1780325053000},"page":"98-107","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Instructor-Centered Per-Question Adaptation for Short Answer Grading Using LLM Scores and Informative Sampling"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-5099-3958","authenticated-orcid":false,"given":"Ovide","family":"Kuichua","sequence":"first","affiliation":[{"name":"Computer Science, Polytechnique Montr\u00e9al, Montr\u00e9al, Qu\u00e9bec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2990-1078","authenticated-orcid":false,"given":"Michel","family":"Desmarais","sequence":"additional","affiliation":[{"name":"Computer Science, Polytechnique Montr\u00e9al, Montr\u00e9al, Qu\u00e9bec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-7828-7708","authenticated-orcid":false,"given":"Arman","family":"Bakhtiari","sequence":"additional","affiliation":[{"name":"Computer Science, Polytechnique Montr\u00e9al, Montr\u00e9al, Qu\u00e9bec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4971-5723","authenticated-orcid":false,"given":"Chahe","family":"Nerguizian","sequence":"additional","affiliation":[{"name":"Electrical Engineering, Polytechnique Montr\u00e9al, Montr\u00e9al, Qu\u00e9bec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,6,7]]},"reference":[{"key":"e_1_3_3_2_2_2","doi-asserted-by":"crossref","unstructured":"Saleema Amershi Maya Cakmak William\u00a0Bradley Knox and Todd Kulesza. 2014. Power to the people: The role of humans in interactive machine learning. AI magazine 35 4 (2014) 105\u2013120.","DOI":"10.1609\/aimag.v35i4.2513"},{"key":"e_1_3_3_2_3_2","unstructured":"Berk Atil Sarp Aykent Alexa Chittams Lisheng Fu Rebecca\u00a0J. Passonneau Evan Radcliffe Guru\u00a0Rajan Rajagopal Adam Sloan Tomasz Tudrej Ferhan Ture Zhe Wu Lixinyu Xu and Breck Baldwin. 2025. Non-Determinism of \"Deterministic\" LLM Settings. arxiv:https:\/\/arXiv.org\/abs\/2408.04667\u00a0[cs.CL] https:\/\/arxiv.org\/abs\/2408.04667"},{"key":"e_1_3_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-84060-0_5"},{"key":"e_1_3_3_2_5_2","doi-asserted-by":"crossref","unstructured":"Sridevi Bonthu S\u00a0Rama Sree and MHM\u00a0Krishna Prasad. 2025. Sprag: building and benchmarking a short programming-related answer grading dataset. International Journal of Data Science and Analytics 20 3 (2025) 1871\u20131883.","DOI":"10.1007\/s41060-024-00576-z"},{"key":"e_1_3_3_2_6_2","unstructured":"Tom Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared\u00a0D Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell et\u00a0al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020) 1877\u20131901."},{"key":"e_1_3_3_2_7_2","doi-asserted-by":"crossref","unstructured":"Peter Brusilovsky. 1999. Adaptive and intelligent technologies for web-based education. technology 2 (1999) 1.","DOI":"10.1007\/3-540-45108-0_1"},{"key":"e_1_3_3_2_8_2","unstructured":"Aubrey Condor Max Litster and Zachary Pardos. 2021. Automatic Short Answer Grading with SBERT on Out-of-Sample Questions. International Educational Data Mining Society (2021)."},{"key":"e_1_3_3_2_9_2","unstructured":"[First name\u00a0missing] Conort. 2012. ASAP \u201912 SAS: Stacking Model for Short Answer Scoring. Kaggle ASAP-SAS\u201912 Competition submission. https:\/\/www.kaggle.com\/c\/asap-sas Uses stacking with 81 base models and OLS for final prediction. QWK = 0.73662. Dataset: ASAP-SAS\u201912.."},{"key":"e_1_3_3_2_10_2","first-page":"503","volume-title":"Proceedings of the 18th International Conference on Educational Data Mining","author":"Desmarais Michel","year":"2025","unstructured":"Michel Desmarais, Arman Bakhtiari, Ovide Kuichua, Samira Chiny\u00a0Folefack Temfack, and Chah\u00e9 Nerguizian. 2025. Short answer grading with sentence similarity and a few given grades. In Proceedings of the 18th International Conference on Educational Data Mining. 503\u2013509."},{"key":"e_1_3_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/EDUCON60312.2024.10578839"},{"key":"e_1_3_3_2_12_2","first-page":"263","volume-title":"Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013)","author":"Dzikovska Myroslava","year":"2013","unstructured":"Myroslava Dzikovska, Rodney Nielsen, Chris Brew, Claudia Leacock, Danilo Giampiccolo, Luisa Bentivogli, Peter Clark, Ido Dagan, and Hoa\u00a0Trang Dang. 2013. SemEval-2013 Task 7: The Joint Student Response Analysis and 8th Recognizing Textual Entailment Challenge. In Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013), Suresh Manandhar and Deniz Yuret (Eds.). Association for Computational Linguistics, Atlanta, Georgia, USA, 263\u2013274. https:\/\/aclanthology.org\/S13-2045\/"},{"key":"e_1_3_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/3706468.3706481"},{"key":"e_1_3_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-long.587"},{"key":"e_1_3_3_2_15_2","unstructured":"Chelsea Finn Pieter Abbeel and Sergey Levine. 2017. Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks. arxiv:https:\/\/arXiv.org\/abs\/1703.03400\u00a0[cs.LG] https:\/\/arxiv.org\/abs\/1703.03400"},{"key":"e_1_3_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3723010.3723021"},{"key":"e_1_3_3_2_17_2","unstructured":"Sasi\u00a0Kiran Gaddipati Deebul Nair and Paul\u00a0G Pl\u00f6ger. 2020. Comparative evaluation of pretrained transfer learning models on automatic short answer grading. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2009.01303 (2020)."},{"key":"e_1_3_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.5753\/sbsi.2020.13133"},{"key":"e_1_3_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/CONIT55038.2022.9847754"},{"key":"e_1_3_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.5220\/0009422400580067"},{"key":"e_1_3_3_2_21_2","volume-title":"Knowing what students know: The science and design of educational assessment","author":"Glaser Robert","year":"2001","unstructured":"Robert Glaser, Naomi Chudowsky, and James\u00a0W Pellegrino. 2001. Knowing what students know: The science and design of educational assessment. National Academies Press."},{"key":"e_1_3_3_2_22_2","doi-asserted-by":"crossref","unstructured":"Wael\u00a0H Gomaa Abdelrahman\u00a0E Nagib Mostafa\u00a0M Saeed Abdulmohsen Algarni and Emad Nabil. 2023. Empowering short answer grading: Integrating transformer-based embeddings and BI-LSTM network. Big Data and Cognitive Computing 7 3 (2023) 122.","DOI":"10.3390\/bdcc7030122"},{"key":"e_1_3_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.5555\/2390384.2390425"},{"key":"e_1_3_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/CIACON65473.2025.11189329"},{"key":"e_1_3_3_2_25_2","unstructured":"Helia Hashemi Jason Eisner Corby Rosset Benjamin Van\u00a0Durme and Chris Kedzie. 2024. LLM-rubric: A multidimensional calibrated approach to automated evaluation of natural language texts. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2501.00274 (2024)."},{"key":"e_1_3_3_2_26_2","unstructured":"Lovish Madaan Aaditya\u00a0K. Singh Rylan Schaeffer Andrew Poulton Sanmi Koyejo Pontus Stenetorp Sharan Narang and Dieuwke Hupkes. 2024. Quantifying Variance in Evaluation Benchmarks. arxiv:https:\/\/arXiv.org\/abs\/2406.10229\u00a0[cs.LG] https:\/\/arxiv.org\/abs\/2406.10229"},{"key":"e_1_3_3_2_27_2","doi-asserted-by":"crossref","unstructured":"Joseph\u00a0P Magliano and Arthur\u00a0C Graesser. 2012. Computer-based assessment of student-constructed responses. Behavior Research Methods 44 3 (2012) 608\u2013621.","DOI":"10.3758\/s13428-012-0211-3"},{"key":"e_1_3_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.5753\/sbie.2024.242424"},{"key":"e_1_3_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.5555\/2002472.2002568"},{"key":"e_1_3_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.3115\/1609067.1609130"},{"key":"e_1_3_3_2_31_2","doi-asserted-by":"crossref","unstructured":"Elizbar\u00a0A Nadaraya. 1964. On estimating regression. Theory of Probability & Its Applications 9 1 (1964) 141\u2013142.","DOI":"10.1137\/1109020"},{"key":"e_1_3_3_2_32_2","doi-asserted-by":"crossref","unstructured":"Leila Ouahrani and Djamal Bennouar. 2024. Paraphrase generation and supervised learning for improved automatic short answer grading. International Journal of Artificial Intelligence in Education 34 4 (2024) 1627\u20131670.","DOI":"10.1007\/s40593-023-00391-w"},{"key":"e_1_3_3_2_33_2","doi-asserted-by":"crossref","unstructured":"Marko Putnikovic and Jelena Jovanovic. 2023. Embeddings for automatic short answer grading: A scoping review. IEEE Transactions on Learning Technologies 16 2 (2023) 219\u2013231.","DOI":"10.1109\/TLT.2023.3253071"},{"key":"e_1_3_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3571662.3571666"},{"key":"e_1_3_3_2_35_2","doi-asserted-by":"crossref","unstructured":"Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/1908.10084 (2019).","DOI":"10.18653\/v1\/D19-1410"},{"key":"e_1_3_3_2_36_2","doi-asserted-by":"crossref","unstructured":"Archana Sahu and Plaban\u00a0Kumar Bhowmick. 2019. Feature engineering and ensemble-based approach for improving automatic short-answer grading performance. IEEE Transactions on Learning Technologies 13 1 (2019) 77\u201390.","DOI":"10.1109\/TLT.2019.2897997"},{"key":"e_1_3_3_2_37_2","doi-asserted-by":"crossref","unstructured":"Shubhra Kanti\u00a0Karmaker Santu and Dongji Feng. 2023. Teler: A general taxonomy of llm prompts for benchmarking complex tasks. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2305.11430 (2023).","DOI":"10.18653\/v1\/2023.findings-emnlp.946"},{"key":"e_1_3_3_2_38_2","volume-title":"Active learning literature survey","author":"Settles Burr","year":"2009","unstructured":"Burr Settles. 2009. Active learning literature survey. Technical Report. University of Wisconsin-Madison Department of Computer Sciences. https:\/\/api.semanticscholar.org\/CorpusID:324600"},{"key":"e_1_3_3_2_39_2","volume-title":"KDD Workshop on Data Mining for Educational Assessment and Feedback (ASSESS 2014)","author":"Shah Nihar\u00a0B","year":"2014","unstructured":"Nihar\u00a0B Shah, Joseph Bradley, Sivaraman Balakrishnan, Abhay Parekh, Kannan Ramchandran, and Martin\u00a0J Wainwright. 2014. Some scaling laws for MOOC assessments. In KDD Workshop on Data Mining for Educational Assessment and Feedback (ASSESS 2014)."},{"key":"e_1_3_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/ITHET61869.2024.10837667"},{"key":"e_1_3_3_2_41_2","unstructured":"Dequan Wang Evan Shelhamer Shaoteng Liu Bruno Olshausen and Trevor Darrell. 2021. Tent: Fully Test-time Adaptation by Entropy Minimization. arxiv:https:\/\/arXiv.org\/abs\/2006.10726\u00a0[cs.LG] https:\/\/arxiv.org\/abs\/2006.10726"},{"key":"e_1_3_3_2_42_2","unstructured":"Yun Wang Zhaojun Ding Xuansheng Wu Siyue Sun Ninghao Liu and Xiaoming Zhai. 2025. AutoSCORE: Enhancing Automated Scoring with Multi-Agent Large Language Models via Structured Component Recognition. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2509.21910 (2025)."},{"key":"e_1_3_3_2_43_2","unstructured":"Geoffrey\u00a0S Watson. 1964. Smooth regression analysis. Sankhy\u0101: The Indian Journal of Statistics Series A (1964) 359\u2013372."},{"key":"e_1_3_3_2_44_2","doi-asserted-by":"crossref","unstructured":"Jason Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Fei Xia Ed Chi Quoc\u00a0V Le Denny Zhou et\u00a0al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems 35 (2022) 24824\u201324837.","DOI":"10.52202\/068431-1800"},{"key":"e_1_3_3_2_45_2","volume-title":"Gaussian processes for machine learning","author":"Williams Christopher\u00a0KI","year":"2006","unstructured":"Christopher\u00a0KI Williams and Carl\u00a0Edward Rasmussen. 2006. Gaussian processes for machine learning. Vol.\u00a02. MIT press Cambridge, MA."},{"key":"e_1_3_3_2_46_2","unstructured":"W Xie J Niu CJ Xue and N Guan. [n. d.]. Grade like a human: rethinking automated assessment with large language models (2024). arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2405.19694 ([n. d.])."},{"key":"e_1_3_3_2_47_2","doi-asserted-by":"crossref","unstructured":"Lixiang Yan Lele Sha Linxuan Zhao Yuheng Li Roberto Martinez-Maldonado Guanliang Chen Xinyu Li Yueqiao Jin and Dragan Ga\u0161evi\u0107. 2024. Practical and ethical challenges of large language models in education: A systematic scoping review. British Journal of Educational Technology 55 1 (2024) 90\u2013112.","DOI":"10.1111\/bjet.13370"},{"key":"e_1_3_3_2_48_2","unstructured":"Jure Zbontar. 2012. ASAP \u201912 SAS: Stacking Method for Short Answer Scoring. Kaggle ASAP-SAS\u201912 Competition submission. https:\/\/www.kaggle.com\/c\/asap-sas Combines BoW representations with ridge regression SVR GBM RF and kNN in a stacking framework. QWK = 0.73892. Dataset: ASAP-SAS\u201912.."},{"key":"e_1_3_3_2_49_2","doi-asserted-by":"crossref","unstructured":"Lishan Zhang Yuwei Huang Xi Yang Shengquan Yu and Fuzhen Zhuang. 2022. An automatic short-answer grading model for semi-open-ended questions. Interactive learning environments 30 1 (2022) 177\u2013190.","DOI":"10.1080\/10494820.2019.1648300"},{"key":"e_1_3_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-98459-4_1"},{"key":"e_1_3_3_2_51_2","doi-asserted-by":"crossref","unstructured":"Xinhua Zhu Han Wu and Lanfang Zhang. 2022. Automatic short-answer grading via BERT-based deep neural networks. IEEE Transactions on Learning Technologies 15 3 (2022) 364\u2013375.","DOI":"10.1109\/TLT.2022.3175537"}],"event":{"name":"UMAP '26: 34th ACM Conference on User Modeling, Adaptation and Personalization","location":"Gothenburg , Sweden","acronym":"UMAP '26","sponsor":["SIGWEB ACM Special Interest Group on Hypertext, Hypermedia, and Web","SIGCHI ACM Special Interest Group on Computer-Human Interaction"]},"container-title":["Proceedings of the 34th ACM Conference on User Modeling, Adaptation and Personalization"],"original-title":[],"deposited":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T15:08:46Z","timestamp":1780326526000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3774935.3806189"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,7]]},"references-count":50,"alternative-id":["10.1145\/3774935.3806189","10.1145\/3774935"],"URL":"https:\/\/doi.org\/10.1145\/3774935.3806189","relation":{},"subject":[],"published":{"date-parts":[[2026,6,7]]},"assertion":[{"value":"2026-06-07","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}