{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,20]],"date-time":"2026-07-20T09:54:02Z","timestamp":1784541242985,"version":"3.55.0"},"reference-count":59,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,4,26]],"date-time":"2021-04-26T00:00:00Z","timestamp":1619395200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,4,26]],"date-time":"2021-04-26T00:00:00Z","timestamp":1619395200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["RPTEL"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Web-based learning systems with adaptive capabilities to personalize content are becoming nowadays a trend in order to offer interactive learning materials to cope with a wide diversity of students attending online education. Learners\u2019 interaction and study practice (quizzing, reading, exams) can be analyzed in order to get some insights into the student\u2019s learning style, study schedule, knowledge, and performance. Quizzing might be used to help to create individualized\/personalized spaced repetition algorithm in order to improve long-term retention of knowledge and provide efficient learning in online learning platforms. Current spaced repetition algorithms have pre-defined repetition rules and parameters that might not be a good fit for students\u2019 different learning styles in online platforms. This study uses different machine learning models and a rich context model to analyze quizzing and reading records from e-learning platform called Hypocampus in order to get some insights into the relevant features to predict learning outcome (quiz answers). By knowing the answer correctness, a learning system might be able to recommend personalized repetitive schedule for questions with maximizing long-term memory retention. Study results show that question difficulty level and incorrectly answered previous questions are useful features to predict the correctness of student\u2019s answer. The gradient-boosted tree and XGBoost models are best in predicting the correctness of the student\u2019s answer before answering a quiz. Additionally, some non-linear relationship was found between the reading learning material behavior in the platform and quiz performance that brings added value to the accuracy for all used models.<\/jats:p>","DOI":"10.1186\/s41039-021-00159-7","type":"journal-article","created":{"date-parts":[[2021,4,26]],"date-time":"2021-04-26T10:06:17Z","timestamp":1619431577000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["The performance of some machine learning approaches and a rich context model in student answer prediction"],"prefix":"10.1186","volume":"16","author":[{"given":"Alisa","family":"Lincke","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marc","family":"Jansen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marcelo","family":"Milrad","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elias","family":"Berge","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,4,26]]},"reference":[{"key":"159_CR1","doi-asserted-by":"publisher","unstructured":"Boyd, K., Eng, K.H., Page, C.D. (2013). Area under the precision-recall curve: Point estimates and confidence intervals. In Joint European conference on machine learning and knowledge discovery in databases. https:\/\/doi.org\/10.1007\/978-3-642-40994-3_29. Springer, (pp. 451\u2013466).","DOI":"10.1007\/978-3-642-40994-3_29"},{"issue":"7","key":"159_CR2","doi-asserted-by":"publisher","first-page":"1145","DOI":"10.1016\/S0031-3203(96)00142-2","volume":"30","author":"A.P. Bradley","year":"1997","unstructured":"Bradley, A.P. (1997). The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern recognition, 30(7), 1145\u20131159.","journal-title":"Pattern recognition"},{"issue":"1","key":"159_CR3","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1002\/(SICI)1097-4571(199401)45:1<12::AID-ASI2>3.0.CO;2-L","volume":"45","author":"M. Buckland","year":"1994","unstructured":"Buckland, M., & Gey, F. (1994). The relationship between recall and precision. Journal of the American society for information science, 45(1), 12\u201319.","journal-title":"Journal of the American society for information science"},{"issue":"3","key":"159_CR4","first-page":"617","volume":"7","author":"M. Bucos","year":"2018","unstructured":"Bucos, M. (2018). Predicting student success using data generated in traditional educational environments. TEM Journal, 7(3), 617.","journal-title":"TEM Journal"},{"key":"159_CR5","doi-asserted-by":"publisher","unstructured":"Chaudhry, R., Singh, H., Dogga, P., Saini, S.K. (2018). Modeling hint-taking behavior and knowledge state of students with multi-task learning. International Educational Data Mining Society. https:\/\/doi.org\/10.29007\/dj6b.","DOI":"10.29007\/dj6b"},{"issue":"1","key":"159_CR6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/1007730.1007733","volume":"6","author":"N.V. Chawla","year":"2004","unstructured":"Chawla, N.V., Japkowicz, N., Kotcz, A. (2004). Special issue on learning from imbalanced data sets. ACM SIGKDD explorations newsletter, 6(1), 1\u20136.","journal-title":"ACM SIGKDD explorations newsletter"},{"issue":"3","key":"159_CR7","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1016\/j.compedu.2004.01.006","volume":"44","author":"C.M. Chen","year":"2005","unstructured":"Chen, C.M., Lee, H.M., Chen, Y.H. (2005). Personalized e-learning system using item response theory. Computers & Education, 44(3), 237\u2013255.","journal-title":"Computers & Education"},{"key":"159_CR8","doi-asserted-by":"publisher","first-page":"785","DOI":"10.1145\/2939672.2939785","volume-title":"Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining","author":"T. Chen","year":"2016","unstructured":"Chen, T., & Guestrin, C. (2016). Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. Association for Computing Machinery New York NY United States, San Francisco California, (pp. 785\u2013794)."},{"issue":"120","key":"159_CR9","first-page":"12","volume":"2020","author":"B. Choffin","year":"2020","unstructured":"Choffin, B., Popineau, F., Bourda, Y. (2020). Modelling student learning and forgetting for optimally scheduling skill review. ERCIM News, 2020(120), 12\u201313.","journal-title":"ERCIM News"},{"key":"159_CR10","doi-asserted-by":"publisher","unstructured":"Chounta, I.A., Albacete, P., Jordan, P., Katz, S., McLaren, B.M. (2017). The \u201cGrey Area\u201d: A computational approach to model the Zone of Proximal Development. In European Conference on Technology Enhanced Learning. https:\/\/doi.org\/10.1007\/978-3-319-66610-5_1. Springer, (pp. 3\u201316).","DOI":"10.1007\/978-3-319-66610-5_1"},{"key":"159_CR11","doi-asserted-by":"publisher","unstructured":"Cieslak, D.A., & Chawla, N.V. (2008). Learning decision trees for unbalanced data. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. https:\/\/doi.org\/10.1007\/978-3-540-87479-9_34. Springer, (pp. 241\u2013256).","DOI":"10.1007\/978-3-540-87479-9_34"},{"key":"159_CR12","doi-asserted-by":"publisher","unstructured":"Coelho, O.B., & Silveira, I. (2017). Deep learning applied to learning analytics and educational data mining: A systematic literature review. In Brazilian Symposium on Computers in Education (Simp\u00f3sio Brasileiro de Inform\u00e1tica na Educa\u00e7\u00e3o-SBIE), vol. 28. https:\/\/doi.org\/10.5753\/cbie.sbie.2017.143, (p. 143).","DOI":"10.5753\/cbie.sbie.2017.143"},{"key":"159_CR13","doi-asserted-by":"crossref","unstructured":"Davis, D., Chen, G., Van der Zee, T., Hau_, C., Houben, G.J. (2016). Retrieval practice and study planning in moocs: Exploring classroombased self-regulated learning strategies at scale. In European conference on technology enhanced learning. Springer, (pp. 57\u201371).","DOI":"10.1007\/978-3-319-45153-4_5"},{"key":"159_CR14","doi-asserted-by":"publisher","unstructured":"Davis, J, & Goadrich, M. (2006). The relationship between precision-recall and ROC curves. In Proceedings of the 23rd international conference on Machine learning. https:\/\/doi.org\/10.1145\/1143844.1143874, (pp. 233\u2013240).","DOI":"10.1145\/1143844.1143874"},{"key":"159_CR15","doi-asserted-by":"crossref","unstructured":"Davis, D., Kizilcec, R.F., Hau_, C., Houben, G.J. (2018). The half-life of mooc knowledge: a randomized trial evaluating knowledge retention and retrieval practice in moocs. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge, (pp. 1\u201310).","DOI":"10.1145\/3170358.3170383"},{"issue":"1","key":"159_CR16","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1177\/1529100612453266","volume":"14","author":"J. Dunlosky","year":"2013","unstructured":"Dunlosky, J., Rawson, K.A., Marsh, E.J., Nathan, M.J., Willingham, D.T. (2013). Improving students\u2019 learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), 4\u201358.","journal-title":"Psychological Science in the Public Interest"},{"key":"159_CR17","unstructured":"Duong, H., Zhu, L., Wang, Y., Heffernan, N.T. (2013). A prediction model that uses the sequence of attempts and hints to better predict knowledge: \u201cBetter to attempt the problem first, rather than ask for a hint\u201d, (pp. 316\u2013317): EDM."},{"issue":"8","key":"159_CR18","doi-asserted-by":"publisher","first-page":"861","DOI":"10.1016\/j.patrec.2005.10.010","volume":"27","author":"T. Fawcett","year":"2006","unstructured":"Fawcett, T. (2006). An introduction to ROC analysis. Pattern recognition letters, 27(8), 861\u2013874.","journal-title":"Pattern recognition letters"},{"key":"159_CR19","doi-asserted-by":"publisher","first-page":"2076","DOI":"10.3389\/fpsyg.2020.02076","volume":"11","author":"D. Fellman","year":"2020","unstructured":"Fellman, D., Lincke, A., Jonsson, B. (2020). Do individual differences in cognition and personality predict retrieval practice activities on moocs?Frontiers in psychology, 11, 2076.","journal-title":"Frontiers in psychology"},{"key":"159_CR20","unstructured":"Galvez, J., Guzman, E., Conejo, R., Millan, E. (2009). Student knowledge diagnosis using item response theory and constraint-based modeling. In Artificial Intelligence in Education (AIED-2009)\u016e Building learning systems that care: from knowledge representation to affective modelling (Vol. 200). IOS Press, (pp. 291\u2013298)."},{"key":"159_CR21","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1016\/j.bdr.2017.05.001","volume":"11","author":"A. Gounaris","year":"2018","unstructured":"Gounaris, A., & Torres, J. (2018). A methodology for spark parameter tuning. Big data research, 11, 22\u201332.","journal-title":"Big data research"},{"key":"159_CR22","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1109\/ISET.2015.33","volume-title":"2015 International Symposium on Educational Technology (ISET)","author":"B. Guo","year":"2015","unstructured":"Guo, B., Zhang, R., Xu, G., Shi, C., Yang, L. (2015). Predicting students performance in educational data mining. In 2015 International Symposium on Educational Technology (ISET). https:\/\/doi.org\/10.1109\/iset.2015.33. Institute of Electrical and Electronics Engineers Inc, IEEE Computer Society, Wuhan, (pp. 125\u2013128)."},{"key":"159_CR23","unstructured":"Hodara, M., Jaggars, S., Karp MJM (2012). Improving developmental education assessment and placement: Lessons from community colleges across the country. (CCRC Working Paper No. 51). New York: Community College Research Center."},{"issue":"Spring","key":"159_CR24","first-page":"104","volume":"22","author":"S.K. House","year":"2016","unstructured":"House, S.K., Sweet, S.L., Vickers, C. (2016). Students\u2019 perceptions and satisfaction with adaptive quizzing. AURCO Journal, 22(Spring), 104\u2013110.","journal-title":"AURCO Journal"},{"key":"159_CR25","unstructured":"Ibrahim, Z, & Rusli, D. (2007). Predicting students\u2019 academic performance: Comparing artificial neural network, decision tree and linear regression. In 21st Annual SAS Malaysia Forum, 5th September, Kuala Lumpur, Malaysia."},{"key":"159_CR26","first-page":"88","volume":"125","author":"E. Joseph","year":"2005","unstructured":"Joseph, E. (2005). Engagement tracing: using response times to model student disengagement. Artificial intelligence in education: Supporting learning through intelligent and socially informed technology, 125, 88.","journal-title":"Artificial intelligence in education: Supporting learning through intelligent and socially informed technology"},{"issue":"5865","key":"159_CR27","doi-asserted-by":"publisher","first-page":"966","DOI":"10.1126\/science.1152408","volume":"319","author":"J.D. Karpicke","year":"2008","unstructured":"Karpicke, J.D., & Roediger, H.L. (2008). The critical importance of retrieval for learning. Science, 319(5865), 966\u2013968.","journal-title":"Science"},{"key":"159_CR28","first-page":"7","volume-title":"Proceedings of Workshop on Personalization Approaches in Learning Environments (PALE 2014) at the 22th International Conference on User Modeling, Adaptation, and Personalization","author":"M.M. Khajah","year":"2014","unstructured":"Khajah, M.M., Huang, Y., Gonz\u00e1lez-Brenes, J.P., Mozer, M.C., Brusilovsky, P. (2014). Integrating knowledge tracing and item response theory: A tale of two frameworks. In Proceedings of Workshop on Personalization Approaches in Learning Environments (PALE 2014) at the 22th International Conference on User Modeling, Adaptation, and Personalization. University of Pittsburgh, Pittsburgh, (pp. 7\u201315)."},{"key":"159_CR29","first-page":"1137","volume-title":"Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence","author":"R. Kohavi","year":"1995","unstructured":"Kohavi, R., & et al (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. In Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence. Morgan Kaufmann, San Francisco, (pp. 1137\u20131143)."},{"issue":"1","key":"159_CR30","first-page":"430","volume":"18","author":"A. Kucukelbir","year":"2017","unstructured":"Kucukelbir, A., Tran, D., Ranganath, R., Gelman, A., Blei, D.M. (2017). Automatic differentiation variational inference. The Journal of Machine Learning Research, 18(1), 430\u2013474.","journal-title":"The Journal of Machine Learning Research"},{"key":"159_CR31","first-page":"42","volume-title":"The 27th International Conference on Computers in Education (Vol. 1)","author":"A Lincke","year":"2019","unstructured":"Lincke, A, Jansen, M, Milrad, M, Berge, E. (2019). Using data mining techniques to assess students\u2019 answer predictions. In The 27th International Conference on Computers in Education (Vol. 1). Asia-Pacific Society for Computers in Education, Kenting, (pp. 42\u201350)."},{"key":"159_CR32","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1016\/j.chb.2017.11.011","volume":"80","author":"J. Maldonado-Mahauad","year":"2018","unstructured":"Maldonado-Mahauad, J., Perez-Sanagustin, M., Kizilcec, R.F., Morales, N., Munoz- Gama, J. (2018). Mining theory-based patterns from big data: Identifying selfregulated learning strategies in massive open online courses. Computers in Human Behavior, 80, 179\u2013196.","journal-title":"Computers in Human Behavior"},{"issue":"1","key":"159_CR33","first-page":"1235","volume":"17","author":"X. Meng","year":"2016","unstructured":"Meng, X., Bradley, J., Yavuz, B., Sparks, E., Venkataraman, S., Liu, D., Freeman, J., Tsai, D., Amde, M., Owen, S., et al (2016). Mllib: machine learning in apache spark. The Journal of Machine Learning Research, 17(1), 1235\u20131241.","journal-title":"The Journal of Machine Learning Research"},{"key":"159_CR34","unstructured":"Mullachery, V., Khera, A., Husain, A. (2018). Bayesian neural networks. arXiv preprint arXiv:180107710."},{"key":"159_CR35","doi-asserted-by":"publisher","first-page":"348","DOI":"10.1007\/978-3-319-19773-9_35","volume-title":"International Conference on Artificial Intelligence in Education","author":"J. Papou\u0161ek","year":"2015","unstructured":"Papou\u0161ek, J., & Pel\u00e1nek, R. (2015). Impact of adaptive educational system behaviour on student motivation. In International Conference on Artificial Intelligence in Education. Springer, Madrid, (pp. 348\u2013357)."},{"key":"159_CR36","doi-asserted-by":"publisher","first-page":"243","DOI":"10.1007\/978-3-642-22362-4_21","volume-title":"International conference on user modeling, adaptation, and personalization","author":"Z.A. Pardos","year":"2011","unstructured":"Pardos, Z.A., & Heffernan, N.T. (2011). KT-IDEM: Introducing item difficulty to the knowledge tracing model. In International conference on user modeling, adaptation, and personalization. Springer, Girona, (pp. 243\u2013254)."},{"issue":"3-5","key":"159_CR37","doi-asserted-by":"publisher","first-page":"313","DOI":"10.1007\/s11257-017-9193-2","volume":"27","author":"R. Pel\u00e1nek","year":"2017","unstructured":"Pel\u00e1nek, R. (2017). Bayesian knowledge tracing, logistic models, and beyond: An overview of learner modeling techniques. User Modeling and User-Adapted Interaction, 27(3-5), 313\u2013350.","journal-title":"User Modeling and User-Adapted Interaction"},{"key":"159_CR38","volume-title":"Machine learning with spark","author":"N. Pentreath","year":"2015","unstructured":"Pentreath, N. (2015). Machine learning with spark. Birmingham: Packt Publishing Ltd."},{"key":"159_CR39","first-page":"505","volume-title":"Advances in neural information processing systems","author":"C. Piech","year":"2015","unstructured":"Piech, C., Bassen, J., Huang, J., Ganguli, S., Sahami, M., Guibas, L.J., Sohl-Dickstein, J. (2015). Deep knowledge tracing. In Advances in neural information processing systems. MIT Press, Montreal, (pp. 505\u2013513)."},{"key":"159_CR40","doi-asserted-by":"publisher","DOI":"10.4324\/9781315736013","volume-title":"Handbook of item response theory modeling: Applications to typical performance assessment","author":"S.P. Reise","year":"2014","unstructured":"Reise, S.P., & Revicki DA. (2014). Handbook of item response theory modeling: Applications to typical performance assessment. Routledge: Taylor & Francis, New York & London."},{"issue":"1","key":"159_CR41","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1016\/j.tics.2010.09.003","volume":"15","author":"H.L. Roediger III","year":"2011","unstructured":"Roediger III, H.L., & Butler, A.C. (2011). The critical role of retrieval practice in long-term retention. Trends in cognitive sciences, 15(1), 20\u201327.","journal-title":"Trends in cognitive sciences"},{"issue":"3","key":"159_CR42","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1111\/j.1467-9280.2006.01693.x","volume":"17","author":"H.L. Roediger III","year":"2006","unstructured":"Roediger III, H.L., & Karpicke, J.D. (2006). Test-enhanced learning: Taking memory tests improves long-term retention. Psychological science, 17(3), 249\u2013255.","journal-title":"Psychological science"},{"issue":"1","key":"159_CR43","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1186\/s41239-018-0113-2","volume":"15","author":"B. Ross","year":"2018","unstructured":"Ross, B., Chase, A.M., Robbie, D., Oates, G., Absalom, Y. (2018). Adaptive quizzes to increase motivation, engagement and learning outcomes in a first year accounting unit. International Journal of Educational Technology in Higher Education, 15(1), 30.","journal-title":"International Journal of Educational Technology in Higher Education"},{"key":"159_CR44","doi-asserted-by":"publisher","unstructured":"Saito, T., & Rehmsmeier, M. (2015). The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLoS ONE, 10(3). https:\/\/doi.org\/10.1371\/journal.pone.0118432.","DOI":"10.1371\/journal.pone.0118432"},{"key":"159_CR45","volume-title":"Linear regression analysis, vol. 329","author":"G.A. Seber","year":"2012","unstructured":"Seber, G.A., & Lee, A.J. (2012). Linear regression analysis, vol. 329. New York: Wiley."},{"key":"159_CR46","unstructured":"Settles, B., & Meeder, B. (2016). A trainable spaced repetition model for language learning. In Proceedings of the 54th annual meeting of the association for computational linguistics (volume 1: long papers), (pp. 1848\u20131858)."},{"key":"159_CR47","doi-asserted-by":"publisher","first-page":"414","DOI":"10.1016\/j.procs.2015.12.157","volume":"72","author":"A.M. Shahiri","year":"2015","unstructured":"Shahiri, A.M., Husain, W., et al (2015). A review on predicting student\u2019s performance using data mining techniques. Procedia Computer Science, 72, 414\u2013422.","journal-title":"Procedia Computer Science"},{"issue":"5","key":"159_CR48","doi-asserted-by":"publisher","first-page":"246","DOI":"10.1097\/NNE.0000000000000258","volume":"41","author":"L. Simon-Campbell","year":"2016","unstructured":"Simon-Campbell, L., Phelan, J., et al (2016). Effectiveness of an adaptive quizzing system as an institutional-wide strategy to improve student learning and retention. Nurse educator, 41(5), 246\u2013251.","journal-title":"Nurse educator"},{"key":"159_CR49","unstructured":"Sotsenko, A. (2017). A rich context model: Design and implementation. PhD thesis, Faculty of Technology, Linnaeus University. V\u00e4xj\u00f6."},{"key":"159_CR50","doi-asserted-by":"crossref","unstructured":"Sotsenko, A., Jansen, M., Milrad, M., Rana, J. (2016a). Using a rich context model for real-time big data analytics in twitter. In 2016 IEEE 4th International Conference on Future Internet of Things and Cloud Workshops. IEEE Computer Society, Vienna, (pp. 228\u2013233).","DOI":"10.1109\/W-FiCloud.2016.55"},{"key":"159_CR51","doi-asserted-by":"crossref","unstructured":"Sotsenko, A., Zbick, J., Jansen, M., Milrad, M. (2016b). Flexible and contextualized cloud applications for mobile learning scenarios. Mobile, ubiquitous, and pervasive learning, 167\u2013192. Springer.","DOI":"10.1007\/978-3-319-26518-6_7"},{"key":"159_CR52","first-page":"2449","volume-title":"Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems","author":"M. Strobel","year":"2019","unstructured":"Strobel, M. (2019). Aspects of transparency in machine learning. In Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems. International Foundation for Autonomous Agents and Multiagent Systems, Richland, (pp. 2449\u20132451)."},{"issue":"10","key":"159_CR53","doi-asserted-by":"publisher","first-page":"3988","DOI":"10.1073\/pnas.1815156116","volume":"116","author":"B. Tabibian","year":"2019","unstructured":"Tabibian, B., Upadhyay, U., De, A., Zarezade, A., Sch\u00f6lkopf, B., Gomez-Rodriguez, M. (2019). Enhancing human learning via spaced repetition optimization. Proceedings of the National Academy of Sciences, 116(10), 3988\u20133993.","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"4","key":"159_CR54","first-page":"1024","volume":"25","author":"K.W. Thiede","year":"1999","unstructured":"Thiede, K.W., & Dunlosky, J. (1999). Toward a general model of self-regulated study: An analysis of selection of items for study and self-paced study time. Journal of experimental psychology: Learning, Memory, and Cognition, 25(4), 1024.","journal-title":"Journal of experimental psychology: Learning, Memory, and Cognition"},{"key":"159_CR55","first-page":"260","volume":"1","author":"K.M. Ting","year":"2010","unstructured":"Ting, K.M. (2010). Confusion Matrix. Encyclopedia of machine learning, 1, 260\u2013260. Springer, Boston.","journal-title":"Encyclopedia of machine learning"},{"issue":"2","key":"159_CR56","doi-asserted-by":"publisher","first-page":"488","DOI":"10.1021\/ci600426e","volume":"47","author":"J.F. Truchon","year":"2007","unstructured":"Truchon, J.F., & Bayly, C.I. (2007). Evaluating virtual screening methods: Good and bad metrics for the \u201cearly recognition\u201d problem. Journal of chemical information and modeling, 47(2), 488\u2013508.","journal-title":"Journal of chemical information and modeling"},{"key":"159_CR57","doi-asserted-by":"crossref","unstructured":"Van der Zee, T., Davis, D., Saab, N., Giesbers, B., Ginn, J., Van Der Sluis, F., Paas, F., Admiraal, W. (2018). Evaluating retrieval practice in a mooc: How writing and reading summaries of videos affects student learning. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge, (pp. 216\u2013225).","DOI":"10.1145\/3170358.3170382"},{"key":"159_CR58","doi-asserted-by":"publisher","unstructured":"Weng, C.G., & Poon, J. (2008). A new evaluation measure for imbalanced datasets. In Proceedings of the 7th Australasian Data Mining Conference-Volume, vol. 87. https:\/\/doi.org\/10.1109\/ijcnn.2011.6033267, (pp. 27\u201332).","DOI":"10.1109\/ijcnn.2011.6033267"},{"key":"159_CR59","unstructured":"Wu, G., & Chang, E.Y. (2003). Class-boundary alignment for imbalanced dataset learning. In ICML 2003 workshop on learning from imbalanced data sets, vol. II, Washington, (pp. 49\u201356)."}],"container-title":["Research and Practice in Technology Enhanced Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s41039-021-00159-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s41039-021-00159-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s41039-021-00159-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,4,26]],"date-time":"2021-04-26T10:35:41Z","timestamp":1619433341000},"score":1,"resource":{"primary":{"URL":"https:\/\/telrp.springeropen.com\/articles\/10.1186\/s41039-021-00159-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,26]]},"references-count":59,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["159"],"URL":"https:\/\/doi.org\/10.1186\/s41039-021-00159-7","relation":{},"ISSN":["1793-7078"],"issn-type":[{"value":"1793-7078","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,26]]},"assertion":[{"value":"29 January 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 April 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 April 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare that they have no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"10"}}