{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T21:27:56Z","timestamp":1768339676258,"version":"3.49.0"},"reference-count":57,"publisher":"Association for Computing Machinery (ACM)","issue":"2","funder":[{"name":"Promotion of Science (JSPS) KAKENHI","award":["JP24K20903"],"award-info":[{"award-number":["JP24K20903"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Knowl. Discov. Data"],"published-print":{"date-parts":[[2026,2,28]]},"abstract":"<jats:p>Language learning applications usually estimate the learner\u2019s language knowledge over time to provide personalized practice content for each learner at the optimal timing. However, accurately predicting language knowledge or linguistic skills is much more challenging than math or science knowledge, as many language tasks involve memorization and retrieval. Learners must memorize a large number of words and meanings, which are prone to be forgotten without practice. Although a few studies consider forgetting when modeling learners\u2019 language knowledge, they tend to apply traditional models, consider only partial information about forgetting, and ignore linguistic features that may significantly influence learning and forgetting. This article focuses on modeling and predicting learners\u2019 knowledge by considering their forgetting behavior and linguistic features in language learning. Specifically, we first explore the existence of forgetting behavior and cross-effects in real-world language learning datasets through empirical studies. Based on these, we propose a model for predicting the probability of recalling a word given a learner\u2019s practice history. The model incorporates (1) three types of key information related to forgetting (time-gap, interaction, and word features), (2) question formats, and (3) similarities between words using the attention mechanism. Extensive experiments on two real-world datasets show that the proposed model improves performance compared to baselines. Moreover, the results indicate that combining multiple types of forgetting information and item format improves performance. In addition, we find that incorporating semantic and morphological features, such as word embeddings, to model similarities between words in a learner\u2019s practice history and their effects on memory also improves the model. Our work indicates a potential future research direction for the knowledge tracing task in second language acquisition, which gives more instructive results for enhancing learning and teaching.<\/jats:p>","DOI":"10.1145\/3778163","type":"journal-article","created":{"date-parts":[[2025,11,27]],"date-time":"2025-11-27T09:20:03Z","timestamp":1764235203000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Integrating Forgetting Behavior and Linguistic Features in Language Learning Models"],"prefix":"10.1145","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1566-880X","authenticated-orcid":false,"given":"Boxuan","family":"Ma","sequence":"first","affiliation":[{"name":"Kyushu University, Fukuoka, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7377-9519","authenticated-orcid":false,"given":"Sora","family":"Fukui","sequence":"additional","affiliation":[{"name":"OpenDNA Inc., Chofu, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7246-3907","authenticated-orcid":false,"given":"Yuji","family":"Ando","sequence":"additional","affiliation":[{"name":"OpenDNA Inc., Chofu, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5831-2152","authenticated-orcid":false,"given":"Shin\u2019ichi","family":"Konomi","sequence":"additional","affiliation":[{"name":"Kyushu University, Fukuoka, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,1,13]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331195"},{"key":"e_1_3_2_3_2","unstructured":"Ghodai Abdelrahman Qing Wang and Bernardo Pereira Nunes. 2022. Knowledge tracing: A survey. arXiv:2201.06953. Retrieved from https:\/\/arxiv.org\/abs\/2201.06953"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.111.4.1036"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.learninstruc.2022.101582"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00051"},{"key":"e_1_3_2_7_2","first-page":"164","volume-title":"International Conference on Intelligent Tutoring Systems","author":"Cen Hao","year":"2006","unstructured":"Hao Cen, Kenneth Koedinger, and Brian Junker. 2006. Learning factors analysis \u2013 A general method for cognitive model evaluation and improvement. In International Conference on Intelligent Tutoring Systems. Springer, 164\u2013175."},{"key":"e_1_3_2_8_2","volume-title":"International Conference on Educational Data Mining (EDM \u201919)","author":"Choffin Beno\u00eet","year":"2019","unstructured":"Beno\u00eet Choffin, Fabrice Popineau, Yolaine Bourda, and Jill-J\u00eann Vie. 2019. DAS3H: Modeling student learning and forgetting for optimally scheduling distributed practice of skills. In International Conference on Educational Data Mining (EDM \u201919)."},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.5214\/ans.0972.7531.200408"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394486.3403282"},{"key":"e_1_3_2_11_2","volume-title":"11th International Conference on Language Resources and Evaluation (LREC \u201918)","author":"Grave \u00c9douard","year":"2018","unstructured":"\u00c9douard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, and Tom\u00e1\u0161 Mikolov. 2018. Learning word vectors for 157 languages. In 11th International Conference on Language Resources and Evaluation (LREC \u201918)."},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/3379507"},{"key":"e_1_3_2_13_2","unstructured":"Mohammad Khajah Robert V. Lindsey and Michael C. Mozer. 2016. How deep is knowledge tracing? In 9th International Conference on Educational Data Mining."},{"key":"e_1_3_2_14_2","first-page":"158","volume-title":"International Conference on Artificial Intelligence in Education","author":"Lalwani Amar","year":"2019","unstructured":"Amar Lalwani and Sweety Agrawal. 2019. What does time tell? Tracing the forgetting curve using deep knowledge tracing. In International Conference on Artificial Intelligence in Education. Springer, 158\u2013162."},{"key":"e_1_3_2_15_2","volume-title":"So Lernt Man Lernen: Angewandte Lernpsychologie \u2013 Ein Weg Zum Erfolg","author":"Leitner Sebastian","year":"1995","unstructured":"Sebastian Leitner. 1995. So Lernt Man Lernen: Angewandte Lernpsychologie \u2013 Ein Weg Zum Erfolg. Herder."},{"key":"e_1_3_2_16_2","volume-title":"42nd International Conference on Machine Learning","author":"Li Ming","year":"2025","unstructured":"Ming Li, Yukang Cheng, Lu Bai, Feilong Cao, Ke Lv, Jiye Liang, and Pietro Lio. 2025. EduLLM: Leveraging large language models and framelet-based signed hypergraph neural networks for student performance prediction. In 42nd International Conference on Machine Learning."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1177\/0956797613504302"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2019.2924374"},{"key":"e_1_3_2_19_2","unstructured":"Qi Liu Shuanghong Shen Zhenya Huang Enhong Chen and Yonghe Zheng. 2021. A survey of knowledge tracing. arXiv:2105.15106. Retrieved from https:\/\/arxiv.org\/abs\/2105.15106"},{"key":"e_1_3_2_20_2","unstructured":"Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy Mike Lewis Luke Zettlemoyer and Veselin Stoyanov. 2019. Roberta: A robustly optimized BERT pretraining approach. arXiv:1907.11692. Retrieved from https:\/\/arxiv.org\/abs\/1907.11692"},{"key":"e_1_3_2_21_2","first-page":"263","volume-title":"International Conference on Artificial Intelligence in Education","author":"Ma Boxuan","year":"2025","unstructured":"Boxuan Ma, Sora Fukui, Yuji Ando, and Shin\u2019ichi Konomi. 2025. Personalized language learning using spaced repetition scheduling. In International Conference on Artificial Intelligence in Education. Springer, 263\u2013276."},{"issue":"1","key":"e_1_3_2_22_2","first-page":"303","article-title":"Investigating concept definition and skill modeling for cognitive diagnosis in language learning","volume":"16","author":"Ma Boxuan","year":"2024","unstructured":"Boxuan Ma, Sora Fukui, Yuji Ando, and Shinichi Konomi. 2024. Investigating concept definition and skill modeling for cognitive diagnosis in language learning. Journal of Educational Data Mining 16, 1 (2024), 303\u2013329.","journal-title":"Journal of Educational Data Mining"},{"key":"e_1_3_2_23_2","first-page":"695","volume-title":"International Conference on Educational Data Mining (EDM \u201922)","author":"Ma Boxuan","year":"2022","unstructured":"Boxuan Ma, Gayan Prasad Hettiarachchi, and Yuji Ando. 2022. Format-Aware item response theory for predicting vocabulary proficiency. In International Conference on Educational Data Mining (EDM \u201922), 695\u2013700."},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3576050.3576062"},{"key":"e_1_3_2_25_2","first-page":"149","volume-title":"16th International Educational Data Mining Society (EDM \u201923)","author":"Ma Boxuan","year":"2023","unstructured":"Boxuan Ma, Gayan Prasad Hettiarachchi, Sora Fukui, and Yuji Ando. 2023. Exploring the effectiveness of vocabulary proficiency diagnosis using linguistic concept and skill modeling. In 16th International Educational Data Mining Society (EDM \u201923), International Educational Data Mining Society, 149\u2013159."},{"key":"e_1_3_2_26_2","first-page":"3111","article-title":"Distributed representations of words and phrases and their compositionality","volume":"26","author":"Mikolov Tomas","year":"2013","unstructured":"Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems, Vol. 26, 3111\u20133179.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_27_2","first-page":"43","volume-title":"Big Data in Cognitive Science","author":"Mozer Michael C.","year":"2016","unstructured":"Michael C. Mozer and Robert V. Lindsey. 2016. Predicting and improving memory retention: Psychological theory matters in the big data era. In Big Data in Cognitive Science. Michael N. Jones (Ed.), Psychology Press, 43\u201373."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313565"},{"key":"e_1_3_2_29_2","first-page":"156","volume-title":"2019 IEEE\/WIC\/ACM International Conference on Web Intelligence (WI)","author":"Nakagawa Hiromi","year":"2019","unstructured":"Hiromi Nakagawa, Yusuke Iwasawa, and Yutaka Matsuo. 2019. Graph-based knowledge tracing: Modeling student proficiency using graph neural network. In 2019 IEEE\/WIC\/ACM International Conference on Web Intelligence (WI). IEEE, 156\u2013163."},{"key":"e_1_3_2_30_2","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9781139524759","volume-title":"Learning Vocabulary in Another Language","author":"ISP Nation","year":"2001","unstructured":"ISP Nation. 2001. Learning Vocabulary in Another Language, Vol. 10, Cambridge university press Cambridge."},{"key":"e_1_3_2_31_2","first-page":"384","volume-title":"12th International Conference on Educational Data Mining (EDM \u201919)","author":"Pandey Shalini","year":"2019","unstructured":"Shalini Pandey and George Karypis. 2019. A self-attentive model for knowledge tracing. In 12th International Conference on Educational Data Mining (EDM \u201919). International Educational Data Mining Society, 384\u2013389."},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340531.3411994"},{"key":"e_1_3_2_33_2","first-page":"1321","article-title":"Predicting the optimal spacing of study: A multiscale context model of memory","volume":"22","author":"Pashler Harold","year":"2009","unstructured":"Harold Pashler, Nicholas Cepeda, Robert V. Lindsey, Ed Vul, and Michael C. Mozer. 2009. Predicting the optimal spacing of study: A multiscale context model of memory. In Advances in Neural Information Processing Systems, Vol. 22, 1321\u20131329.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_34_2","unstructured":"Philip I. Pavlik Jr Hao Cen and Kenneth R. Koedinger. 2009. Performance factors analysis \u2013 A new alternative to knowledge tracing. In Frontiers in Artificial Intelligence and Applications 531\u2013538."},{"key":"e_1_3_2_35_2","volume-title":"8th International Conference on Educational Data Mining (EDM \u201915)","author":"Pel\u00e1nek Radek","year":"2015","unstructured":"Radek Pel\u00e1nek. 2015. Modeling students\u2019 memory for application in adaptive educational systems. In 8th International Conference on Educational Data Mining (EDM \u201915). International Educational Data Mining Society."},{"key":"e_1_3_2_36_2","doi-asserted-by":"crossref","first-page":"125","DOI":"10.4324\/9780429291586-9","volume-title":"The Routledge Handbook of Vocabulary Studies","author":"Peters Elke","year":"2019","unstructured":"Elke Peters. 2019. Factors affecting the learning of single-word items. In The Routledge Handbook of Vocabulary Studies. Stuart Webb (Ed.), Routledge, 125\u2013142."},{"key":"e_1_3_2_37_2","first-page":"505","article-title":"Deep knowledge tracing","volume":"28","author":"Piech Chris","year":"2015","unstructured":"Chris Piech, Jonathan Bassen, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas J. Guibas, and Jascha Sohl-Dickstein. 2015. Deep knowledge tracing. In Advances in Neural Information Processing Systems, Vol. 28, 505\u2013513.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1540-4781.1967.tb06700.x"},{"key":"e_1_3_2_39_2","first-page":"139","volume-title":"4th International Conference on Educational Data Mining (EDM \u201911)","author":"Qiu Yumeng","year":"2011","unstructured":"Yumeng Qiu, Yingmei Qi, Hanyuan Lu, Zachary A. Pardos, and Neil T. Heffernan. 2011. Does time matter? Modeling the effect of time with bayesian knowledge tracing. In 4th International Conference on Educational Data Mining (EDM \u201911), 139\u2013148."},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3706468.3706501"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_3_2_42_2","doi-asserted-by":"crossref","first-page":"1848","DOI":"10.18653\/v1\/P16-1174","volume-title":"54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Settles Burr","year":"2016","unstructured":"Burr Settles and Brendan Meeder. 2016. A trainable spaced repetition model for language learning. In 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 1848\u20131858."},{"issue":"8","key":"e_1_3_2_43_2","first-page":"8213","article-title":"Monitoring student progress for learning process-consistent knowledge tracing","volume":"35","author":"Shen Shuanghong","year":"2022","unstructured":"Shuanghong Shen, Enhong Chen, Qi Liu, Zhenya Huang, Wei Huang, Yu Yin, Yu Su, and Shijin Wang. 2022. Monitoring student progress for learning process-consistent knowledge tracing. IEEE Transactions on Knowledge and Data Engineering 35, 8 (2022), 8213\u20138227.","journal-title":"IEEE Transactions on Knowledge and Data Engineering"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3447548.3467237"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2023.3251721"},{"key":"e_1_3_2_46_2","volume-title":"32nd AAAI Conference on Artificial Intelligence and 13th Innovative Applications of Artificial Intelligence Conference and 8th AAAI Symposium on Educational Advances in Artificial Intelligence (AAI \u201918\/IAAI \u201918\/EAAI \u201918)","author":"Su Yu","year":"2018","unstructured":"Yu Su, Qingwen Liu, Qi Liu, Zhenya Huang, Yu Yin, Enhong Chen, Chris Ding, Si Wei, and Guoping Hu. 2018. Exercise-enhanced sequential modeling for student performance prediction. In 32nd AAAI Conference on Artificial Intelligence and 13th Innovative Applications of Artificial Intelligence Conference and 8th AAAI Symposium on Educational Advances in Artificial Intelligence (AAI \u201918\/IAAI \u201918\/EAAI \u201918)."},{"issue":"3","key":"e_1_3_2_47_2","doi-asserted-by":"crossref","first-page":"559","DOI":"10.1111\/lang.12343","article-title":"The effects of repetition on incidental vocabulary learning: A meta-analysis of correlational studies","volume":"69","author":"Uchihara Takumi","year":"2019","unstructured":"Takumi Uchihara, Stuart Webb, and Akifumi Yanagisawa. 2019. The effects of repetition on incidental vocabulary learning: A meta-analysis of correlational studies. Language Learning 69, 3 (2019), 559\u2013599.","journal-title":"Language Learning"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4757-2691-6"},{"key":"e_1_3_2_49_2","first-page":"5998","article-title":"Attention is all you need","volume":"30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in Neural Information Processing Systems 30, 5998\u20136008.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3437963.3441802"},{"issue":"4","key":"e_1_3_2_51_2","doi-asserted-by":"crossref","first-page":"715","DOI":"10.1111\/modl.12671","article-title":"How effective are intentional vocabulary-learning activities? A meta-analysis","volume":"104","author":"Webb Stuart","year":"2020","unstructured":"Stuart Webb, Akifumi Yanagisawa, and Takumi Uchihara. 2020. How effective are intentional vocabulary-learning activities? A meta-analysis. The Modern Language Journal 104, 4 (2020), 715\u2013738.","journal-title":"The Modern Language Journal"},{"key":"e_1_3_2_52_2","first-page":"374","volume-title":"13th Workshop on Innovative Use of NLP for Building Educational Applications","author":"Xu Shuyao","year":"2018","unstructured":"Shuyao Xu, Jin Chen, and Long Qin. 2018. CLUF: A neural model for second language acquisition modeling. In 13th Workshop on Innovative Use of NLP for Building Educational Applications, 374\u2013380."},{"issue":"5","key":"e_1_3_2_53_2","doi-asserted-by":"crossref","first-page":"1279","DOI":"10.1017\/S0272263121000577","article-title":"Involvement load hypothesis plus: Creating an improved predictive model of incidental vocabulary learning","volume":"44","author":"Yanagisawa Akifumi","year":"2022","unstructured":"Akifumi Yanagisawa and Stuart Webb. 2022. Involvement load hypothesis plus: Creating an improved predictive model of incidental vocabulary learning. Studies in Second Language Acquisition 44, 5 (2022), 1279\u20131308.","journal-title":"Studies in Second Language Acquisition"},{"key":"e_1_3_2_54_2","first-page":"4381","volume-title":"28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","author":"Ye Junyao","year":"2022","unstructured":"Junyao Ye, Jingyong Su, and Yilong Cao. 2022. A stochastic shortest path algorithm for optimizing spaced repetition scheduling. In 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 4381\u20134390."},{"key":"e_1_3_2_55_2","first-page":"358","volume-title":"International Conference on Artificial Intelligence in Education","author":"Zaidi Ahmed","year":"2020","unstructured":"Ahmed Zaidi, Andrew Caines, Russell Moore, Paula Buttery, and Andrew Rice. 2020. Adaptive forgetting curves for spaced repetition language learning. In International Conference on Artificial Intelligence in Education. Springer, 358\u2013363."},{"key":"e_1_3_2_56_2","first-page":"177","volume-title":"International Conference on Artificial Intelligence in Education","author":"Zhan Bojun","year":"2024","unstructured":"Bojun Zhan, Teng Guo, Xueyi Li, Mingliang Hou, Qianru Liang, Boyu Gao, Weiqi Luo, and Zitao Liu. 2024. Knowledge tracing as language processing: A large-scale autoregressive paradigm. In International Conference on Artificial Intelligence in Education. Springer, 177\u2013191."},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3038912.3052580"},{"key":"e_1_3_2_58_2","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1145\/3448139.3448153","volume-title":"11th International Learning Analytics and Knowledge Conference (LAK \u201921)","author":"Zylich Brian","year":"2021","unstructured":"Brian Zylich and Andrew Lan. 2021. Linguistic skill modeling for second language acquisition. In 11th International Learning Analytics and Knowledge Conference (LAK \u201921), 141\u2013150."}],"container-title":["ACM Transactions on Knowledge Discovery from Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3778163","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T14:54:46Z","timestamp":1768316086000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3778163"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,13]]},"references-count":57,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,2,28]]}},"alternative-id":["10.1145\/3778163"],"URL":"https:\/\/doi.org\/10.1145\/3778163","relation":{},"ISSN":["1556-4681","1556-472X"],"issn-type":[{"value":"1556-4681","type":"print"},{"value":"1556-472X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,13]]},"assertion":[{"value":"2024-01-31","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-11-13","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-01-13","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}