{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T10:50:32Z","timestamp":1775213432509,"version":"3.50.1"},"reference-count":33,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,4,29]],"date-time":"2025-04-29T00:00:00Z","timestamp":1745884800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,4,29]],"date-time":"2025-04-29T00:00:00Z","timestamp":1745884800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100004721","name":"The University of Tokyo","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100004721","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Tech Know Learn"],"published-print":{"date-parts":[[2026,3]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Knowledge tracing (KT) refers to the process of efficiently tracking student achievement in online learning systems. Bayesian knowledge tracing (BKT) is a representative statistical model used for this process. Despite the widespread application of BKT in predicting student performance and modeling knowledge acquisition, methods to evaluate the consistency of measurement by BKT models remain elusive. In psychometrics,\n                    <jats:italic>reliability<\/jats:italic>\n                    is a fundamental concept that gauges the degree to which an assessment tool generates stable and consistent results. Evaluation of reliability is crucial because, without reliable measurement, the adequacy of educational assessments and the decisions based on them are compromised. To address the lack of a method to measure reliability, we propose a novel approach for estimating the reliability coefficient by extending the existing method for diagnostic classification models to time-series data. We apply the proposed method to actual response data and demonstrate its capability of assessing the reliability of BKT models, highlighting the importance of evaluating reliability and its potential for improving individualized learning experiences.\n                  <\/jats:p>","DOI":"10.1007\/s10758-025-09829-7","type":"journal-article","created":{"date-parts":[[2025,4,29]],"date-time":"2025-04-29T03:43:40Z","timestamp":1745898220000},"page":"585-604","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Reliability Coefficient for Bayesian Knowledge Tracing Models"],"prefix":"10.1007","volume":"31","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-3506-4274","authenticated-orcid":false,"given":"Daisuke","family":"Shimada","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1663-5812","authenticated-orcid":false,"given":"Kensuke","family":"Okada","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,4,29]]},"reference":[{"issue":"11","key":"9829_CR1","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3569576","volume":"55","author":"G Abdelrahman","year":"2023","unstructured":"Abdelrahman, G., Wang, Q., & Nunes, B. (2023). Knowledge tracing: A Survey. ACM Computing Surveys, 55(11), 1\u201337. https:\/\/doi.org\/10.1145\/3569576","journal-title":"ACM Computing Surveys"},{"key":"9829_CR2","unstructured":"American Educational Research Association, American Psychological Association, & National Council on Measurement in. (2014). In Education (Ed.), Standards for educational and psychological testing. American Educational Research Association."},{"issue":"1","key":"9829_CR3","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1016\/0004-3702(90)90093-F","volume":"42","author":"JR Anderson","year":"1990","unstructured":"Anderson, J. R., Boyle, C. F., Corbett, A. T., & Lewis, M. W. (1990). Cognitive modeling and intelligent tutoring. Artificial Intelligence, 42(1), 7\u201349. https:\/\/doi.org\/10.1016\/0004-3702(90)90093-F","journal-title":"Artificial Intelligence"},{"key":"9829_CR4","unstructured":"Badrinath, A., Wang, F., & Pardos, Z. (2021). pyBKT: An Accessible Python Library of Bayesian Knowledge Tracing Models. https:\/\/arxiv.org\/abs\/2105.00385v2"},{"key":"9829_CR5","doi-asserted-by":"publisher","unstructured":"Beck, J. E., & Chang, K. (2007). Identifiability: A Fundamental Problem of Student Modeling. In C. Conati, K. McCoy, & G. Paliouras (Eds.), User Modeling 2007 (pp. 137\u2013146). Springer. https:\/\/doi.org\/10.1007\/978-3-540-73078-1_17","DOI":"10.1007\/978-3-540-73078-1_17"},{"key":"9829_CR6","doi-asserted-by":"publisher","unstructured":"Bhatt, S., Zhao, J., Thille, C., Zimmaro, D., & Gattani, N. (2020). Evaluating bayesian knowledge tracing for estimating Learner Proficiency and Guiding Learner Behavior. Proceedings of the Seventh ACM Conference on Learning @ Scale, 357\u2013360. https:\/\/doi.org\/10.1145\/3386527.3406746","DOI":"10.1145\/3386527.3406746"},{"issue":"4","key":"9829_CR7","doi-asserted-by":"publisher","first-page":"253","DOI":"10.1007\/BF01099821","volume":"4","author":"AT Corbett","year":"1994","unstructured":"Corbett, A. T., & Anderson, J. R. (1994). Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4(4), 253\u2013278. https:\/\/doi.org\/10.1007\/BF01099821","journal-title":"User Modeling and User-Adapted Interaction"},{"issue":"2","key":"9829_CR8","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1007\/BF02289001","volume":"18","author":"MD Davidoff","year":"1953","unstructured":"Davidoff, M. D., & Goheen, H. W. (1953). A table for the rapid determination of the tetrachoric correlation coefficient. Psychometrika, 18(2), 115\u2013121. https:\/\/doi.org\/10.1007\/BF02289001","journal-title":"Psychometrika"},{"issue":"2","key":"9829_CR9","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1007\/BF02293968","volume":"44","author":"DR Divgi","year":"1979","unstructured":"Divgi, D. R. (1979). Calculation of the tetrachoric correlation coefficient. Psychometrika, 44(2), 169\u2013172. https:\/\/doi.org\/10.1007\/BF02293968","journal-title":"Psychometrika"},{"key":"9829_CR10","doi-asserted-by":"publisher","unstructured":"Doroudi, S. (2020). Mastery Learning Heuristics and Their Hidden Models. In I. I. Bittencourt, M. Cukurova, K. Muldner, R. Luckin, & E. Mill\u00e1n (Eds.), Artificial Intelligence in Education (pp. 86\u201391). Springer International Publishing. https:\/\/doi.org\/10.1007\/978-3-030-52240-7_16","DOI":"10.1007\/978-3-030-52240-7_16"},{"issue":"3","key":"9829_CR11","doi-asserted-by":"publisher","first-page":"243","DOI":"10.1007\/s11257-009-9063-7","volume":"19","author":"M Feng","year":"2009","unstructured":"Feng, M., Heffernan, N., & Koedinger, K. (2009). Addressing the assessment challenge with an online system that tutors as it assesses. User Modeling and User-Adapted Interaction, 19(3), 243\u2013266. https:\/\/doi.org\/10.1007\/s11257-009-9063-7","journal-title":"User Modeling and User-Adapted Interaction"},{"key":"9829_CR12","unstructured":"Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2014). Multivariate data analysis (Seventh edition, Pearson new international edition). Pearson Education Limited; WorldCat."},{"issue":"4","key":"9829_CR13","doi-asserted-by":"publisher","first-page":"523","DOI":"10.1177\/00131640021970691","volume":"60","author":"TP Hogan","year":"2000","unstructured":"Hogan, T. P., Benjamin, A., & Brezinski, K. L. (2000). Reliability methods: A note on the frequency of Use of various types. Educational and Psychological Measurement, 60(4), 523\u2013531. https:\/\/doi.org\/10.1177\/00131640021970691","journal-title":"Educational and Psychological Measurement"},{"issue":"1","key":"9829_CR14","doi-asserted-by":"publisher","first-page":"5","DOI":"10.3102\/1076998619864550","volume":"45","author":"MS Johnson","year":"2020","unstructured":"Johnson, M. S., & Sinharay, S. (2020). The reliability of the posterior probability of Skill Attainment in Diagnostic classification models. Journal of Educational and Behavioral Statistics, 45(1), 5\u201331. https:\/\/doi.org\/10.3102\/1076998619864550","journal-title":"Journal of Educational and Behavioral Statistics"},{"issue":"4","key":"9829_CR15","doi-asserted-by":"publisher","first-page":"450","DOI":"10.1109\/TLT.2017.2689017","volume":"10","author":"T Kaser","year":"2017","unstructured":"Kaser, T., Klingler, S., Schwing, A. G., & Gross, M. (2017). Dynamic bayesian networks for student modeling. IEEE Transactions on Learning Technologies, 10(4), 450\u2013462. https:\/\/doi.org\/10.1109\/TLT.2017.2689017","journal-title":"IEEE Transactions on Learning Technologies"},{"key":"9829_CR16","unstructured":"Khajah, M., Lindsey, R. V., & Mozer, M. C. (2016). How deep is knowledge tracing? (arXiv:1604.02416). arXiv. http:\/\/arxiv.org\/abs\/1604.02416"},{"key":"9829_CR17","unstructured":"Liu, Q., Shen, S., Huang, Z., Chen, E., & Zheng, Y. (2021). A Survey of Knowledge Tracing (arXiv:2105.15106). arXiv. http:\/\/arxiv.org\/abs\/2105.15106"},{"key":"9829_CR18","doi-asserted-by":"publisher","unstructured":"Naeem, M., Tidswell, A., & Magdy, Y. (2021). Bayesian Knowledge Tracing for Assessment Results Analysis. 2021 17th International Computer Engineering Conference (ICENCO), 30-34. https:\/\/doi.org\/10.1109\/ICENCO49852.2021.9698953","DOI":"10.1109\/ICENCO49852.2021.9698953"},{"key":"9829_CR19","doi-asserted-by":"publisher","unstructured":"Pardos, Z. A., & Heffernan, N. T. (2010). Modeling individualization in a Bayesian network implementation of knowledge tracing. Proceedings of the 18th International Conference on User Modeling, Adaptation, and Personalization, 255\u2013266. https:\/\/doi.org\/10.1007\/978-3-642-13470-8_24","DOI":"10.1007\/978-3-642-13470-8_24"},{"key":"9829_CR20","doi-asserted-by":"publisher","unstructured":"Pardos, Z. A., & Heffernan, N. T. (2011). KT-IDEM: Introducing item difficulty to the knowledge tracing model. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 6787 LNCS, 243\u2013254. https:\/\/doi.org\/10.1007\/978-3-642-22362-4_21","DOI":"10.1007\/978-3-642-22362-4_21"},{"key":"9829_CR222","doi-asserted-by":"crossref","unstructured":"Pardos, Z.A., Baker, R.S.J.d., San Pedro, M.O.C.Z., Gowda, S.M., Gowda, S.M. (2013) Affective states and state tests: Investigating how affect throughout the school year predicts end of year learning outcomes. Proceedings of the 3rd International Conference on Learning Analytics and Knowledge, 117\u2013124.","DOI":"10.1145\/2460296.2460320"},{"issue":"262\u2013273","key":"9829_CR21","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1098\/rsta.1900.0022","volume":"195","author":"K Pearson","year":"1997","unstructured":"Pearson, K. (1997). I. Mathematical contributions to the theory of evolution. \u2014VII. On the correlation of characters not quantitatively measurable. Philosophical Transactions of the Royal Society of London Series A Containing Papers of a Mathematical or Physical Character, 195(262\u2013273), 1\u201347. https:\/\/doi.org\/10.1098\/rsta.1900.0022","journal-title":"Philosophical Transactions of the Royal Society of London Series A Containing Papers of a Mathematical or Physical Character"},{"key":"9829_CR22","doi-asserted-by":"publisher","unstructured":"Pel\u00e1nek, R. (2018). Conceptual Issues in Mastery Criteria: Differentiating Uncertainty and Degrees of Knowledge. In C. Penstein Ros\u00e9, R. Mart\u00ednez-Maldonado, H. U. Hoppe, R. Luckin, M. Mavrikis, K. Porayska-Pomsta, B. McLaren, & B. Du Boulay (Eds.), Artificial Intelligence in Education (Vol. 10947, pp. 450\u2013461). Springer International Publishing. https:\/\/doi.org\/10.1007\/978-3-319-93843-1_33","DOI":"10.1007\/978-3-319-93843-1_33"},{"key":"9829_CR23","unstructured":"R Core Team. (2022). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing. https:\/\/www.R-project.org\/"},{"key":"9829_CR24","unstructured":"Rupp, A. A., Templin, J., & Henson, R. A. (2010). Diagnostic measurement: Theory, methods, and applications. Guilford Press."},{"issue":"2","key":"9829_CR25","doi-asserted-by":"publisher","first-page":"475","DOI":"10.1007\/s41237-018-0072-x","volume":"45","author":"S Slater","year":"2018","unstructured":"Slater, S., & Baker, R. S. (2018). Degree of error in bayesian knowledge tracing estimates from differences in sample sizes. Behaviormetrika, 45(2), 475\u2013493. https:\/\/doi.org\/10.1007\/s41237-018-0072-x","journal-title":"Behaviormetrika"},{"key":"9829_CR26","unstructured":"Stamper, J., Niculescu-Mizil, A., Ritter, S., Gordon, G. J., & Koedinger, K. R. (2010a). Algebra I 2005\u20132006. Development data set from KDD Cup 2010 Educational Data Mining Challenge. http:\/\/pslcdatashop.web.cmu.edu\/KDDCup\/downloads.jsp"},{"key":"9829_CR27","unstructured":"Stamper, J., Niculescu-Mizil, A., Ritter, S., Gordon, G. J., & Koedinger, K. R. (2010b). Bridge to Algebra 2006\u20132007. Development data set from KDD Cup 2010 Educational Data Mining Challenge. http:\/\/pslcdatashop.web.cmu.edu\/KDDCup\/downloads.jsp"},{"key":"9829_CR28","doi-asserted-by":"publisher","unstructured":"Takami, K., Dai, Y., Flanagan, B., & Ogata, H. (2022). Educational Explainable Recommender Usage and its Effectiveness in High School Summer Vacation Assignment. LAK22: 12th International Learning Analytics and Knowledge Conference, 458\u2013464. https:\/\/doi.org\/10.1145\/3506860.3506882","DOI":"10.1145\/3506860.3506882"},{"issue":"2","key":"9829_CR29","doi-asserted-by":"publisher","first-page":"251","DOI":"10.1007\/S00357-013-9129-4","volume":"30","author":"J Templin","year":"2013","unstructured":"Templin, J., & Bradshaw, L. (2013). Measuring the reliability of diagnostic classification Model Examinee estimates. Journal of Classification, 30(2), 251\u2013275. https:\/\/doi.org\/10.1007\/S00357-013-9129-4","journal-title":"Journal of Classification"},{"key":"9829_CR30","unstructured":"Van Rossum, G., & Drake, F. L. (2009). Python 3 reference Manual. CreateSpace."},{"key":"9829_CR31","doi-asserted-by":"crossref","unstructured":"VandenBos, G. R. (Ed.). (2015). APA dictionary of psychology (2nd ed.). American Psychological Association.","DOI":"10.1037\/14646-000"},{"key":"9829_CR32","doi-asserted-by":"publisher","unstructured":"Yudelson, M. V., Koedinger, K. R., & Gordon, G. J. (2013). Individualized Bayesian knowledge tracing models. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 7926 LNAI, 171\u2013180. https:\/\/doi.org\/10.1007\/978-3-642-39112-5_18","DOI":"10.1007\/978-3-642-39112-5_18"}],"container-title":["Technology, Knowledge and Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10758-025-09829-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10758-025-09829-7","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10758-025-09829-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,3]],"date-time":"2026-04-03T10:01:12Z","timestamp":1775210472000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10758-025-09829-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,29]]},"references-count":33,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,3]]}},"alternative-id":["9829"],"URL":"https:\/\/doi.org\/10.1007\/s10758-025-09829-7","relation":{},"ISSN":["2211-1662","2211-1670"],"issn-type":[{"value":"2211-1662","type":"print"},{"value":"2211-1670","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,29]]},"assertion":[{"value":"3 February 2025","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 April 2025","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no relevant financial or non-financial interests to disclose.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing Interests"}}]}}