{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T12:38:41Z","timestamp":1773837521943,"version":"3.50.1"},"reference-count":82,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T00:00:00Z","timestamp":1773792000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T00:00:00Z","timestamp":1773792000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Smart Learn. Environ."],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Learning engagement is inherently dynamic, historically dependent, and oriented to long-term utility. Building on sequential decision-making theory, this study simulates the dynamic, long-term evolution of online learning engagement among Chinese EFL learners using reinforcement learning models for the first time. The findings unveil that online learning engagement can be reconceptualized as a sequential decision-making process. In the process, three optimal policy regimes, stable reinforcement strategies, conservative strategies, and proactive strategies, are found. They can explain how learning engagement is sustained, stabilized, or redirected across various learners\u2019 profiles. Furthermore, learner attributes such as online self-efficacy, metacognitive engagement, and social interactions, jointly reconfigure the sequential decision-making process, altering state representations, rewards, and action costs, thereby generating divergent optimal decision-making strategies. Therein, social interaction is the most powerful predictor. Despite its intrinsic limitations, the present study underscores the essential nature of language learning. More critically, it proposes an innovative interdisciplinary framework for SLA research, which synthesizes insights from decision science, cognitive science, machine learning, and SLA itself.<\/jats:p>","DOI":"10.1186\/s40561-026-00445-1","type":"journal-article","created":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T09:17:06Z","timestamp":1773825426000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Sequential decision-making in online learning engagement among Chinese EFL learners: a reinforcement learning approach"],"prefix":"10.1186","volume":"13","author":[{"given":"Hong","family":"Xie","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2026,3,18]]},"reference":[{"key":"445_CR1","doi-asserted-by":"publisher","unstructured":"Alsahil, A., Abdel Latif, M. M. M., & Alsuhaibani, Z. (2024). Exploring EFL learner engagement with different teacher feedback modes. Computer Assisted Language Learning, 1\u201328. https:\/\/doi.org\/10.1080\/09588221.2024.2374785","DOI":"10.1080\/09588221.2024.2374785"},{"issue":"9","key":"445_CR2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.patter.2021.100316","volume":"2","author":"P Ammanabrolu","year":"2021","unstructured":"Ammanabrolu, P., & Riedl, M. O. (2021). Situated language learning via interactive narratives. Patterns, 2(9), 1\u20138. https:\/\/doi.org\/10.1016\/j.patter.2021.100316","journal-title":"Patterns"},{"issue":"4","key":"445_CR3","doi-asserted-by":"publisher","first-page":"818","DOI":"10.1007\/s11031-013-9346-5","volume":"37","author":"V Bagneux","year":"2013","unstructured":"Bagneux, V., Font, H., & Bollon, T. (2013). Incidental emotions associated with uncertainty appraisals impair decisions in the Iowa Gambling Task. Motivation and Emotion, 37(4), 818\u2013827. https:\/\/doi.org\/10.1007\/s11031-013-9346-5","journal-title":"Motivation and Emotion"},{"key":"445_CR4","first-page":"23","volume-title":"Social Foundations of Thought and Action","author":"A Bandura","year":"1986","unstructured":"Bandura, A. (1986). Social Foundations of Thought and Action (pp. 23\u201328). Prentice Hall."},{"issue":"5","key":"445_CR5","first-page":"679","volume":"6","author":"R Bellman","year":"1957","unstructured":"Bellman, R. (1957). A markovian decision process. Journal of Mathematics and Mechanics, 6(5), 679\u2013684.","journal-title":"Journal of Mathematics and Mechanics"},{"issue":"1","key":"445_CR6","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5\u201332. https:\/\/doi.org\/10.1023\/A:1010933404324","journal-title":"Machine Learning"},{"issue":"5","key":"445_CR7","doi-asserted-by":"publisher","first-page":"926","DOI":"10.1037\/a0026788","volume":"12","author":"M Cassotti","year":"2012","unstructured":"Cassotti, M., Habib, M., Poirel, N., A\u00efte, A., Houd\u00e9, O., & Moutier, S. (2012). Positive emotional context eliminates the framing effect in decision-making. Emotion, 12(5), 926\u2013931. https:\/\/doi.org\/10.1037\/a0026788","journal-title":"Emotion"},{"issue":"4","key":"445_CR8","doi-asserted-by":"publisher","first-page":"3373","DOI":"10.1609\/aaai.v34i04.5739","volume":"34","author":"Y Chandak","year":"2020","unstructured":"Chandak, Y., Theocharous, G., Nota, C., & Thomas, P. (2020). Lifelong learning with a changing action set. In Proceedings of the AAAI Conference on Artificial Intelligence, 34(4), 3373\u20133380. https:\/\/doi.org\/10.1609\/aaai.v34i04.5739","journal-title":"In Proceedings of the AAAI Conference on Artificial Intelligence"},{"issue":"3","key":"445_CR9","doi-asserted-by":"publisher","DOI":"10.1111\/jcal.70030","volume":"41","author":"YS Chang","year":"2025","unstructured":"Chang, Y. S. (2025). Engagement, academic emotions, and artificial intelligence application creativity in the metaverse: A study of facial expressions. Journal of Computer Assisted Learning, 41(3), Article e70030. https:\/\/doi.org\/10.1111\/jcal.70030","journal-title":"Journal of Computer Assisted Learning"},{"issue":"4","key":"445_CR10","first-page":"33","volume":"22","author":"CH Chen","year":"2019","unstructured":"Chen, C. H., & Su, C. Y. (2019). Using the BookRoll e-book system to promote self-regulated learning, self efficacy and academic achievement for university students. Journal of Educational Technology & Society, 22(4), 33\u201346.","journal-title":"Journal of Educational Technology & Society"},{"key":"445_CR11","doi-asserted-by":"publisher","first-page":"83","DOI":"10.3969\/j.issn.1674-8921.2023.05.008","volume":"5","author":"D Chen","year":"2023","unstructured":"Chen, D., & Deng, Y. (2023). Metacognitive engagement of Chinese college students in English deep reading: An empirical study based on digital footprint. Contemporary Foreign Language Studies, 5, 83\u201392. https:\/\/doi.org\/10.3969\/j.issn.1674-8921.2023.05.008","journal-title":"Contemporary Foreign Language Studies"},{"issue":"4","key":"445_CR12","doi-asserted-by":"publisher","first-page":"219","DOI":"10.1080\/00461520.2014.965823","volume":"49","author":"MTH Chi","year":"2014","unstructured":"Chi, M. T. H., & Wylie, R. (2014). The ICAP framework: Linking cognitive engagement to active learning outcomes. Educational Psychologist, 49(4), 219\u2013243. https:\/\/doi.org\/10.1080\/00461520.2014.965823","journal-title":"Educational Psychologist"},{"issue":"167","key":"445_CR13","first-page":"1","volume":"18","author":"Y Chow","year":"2018","unstructured":"Chow, Y., Ghavamzadeh, M., Janson, L., & Pavone, M. (2018). Risk-constrained reinforcement learning with percentile risk criteria. Journal of Machine Learning Research, 18(167), 1\u201351.","journal-title":"Journal of Machine Learning Research"},{"key":"445_CR14","doi-asserted-by":"publisher","DOI":"10.4324\/9780203969465","volume-title":"Student Engagement in Campus-Based and Online Education: University Connections","author":"H Coates","year":"2006","unstructured":"Coates, H. (2006). Student Engagement in Campus-Based and Online Education: University Connections. Routledge."},{"issue":"1","key":"445_CR15","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1017\/S1366728906002732","volume":"10","author":"K de Bot","year":"2007","unstructured":"de Bot, K., Lowie, W., & Verspoor, M. (2007). A dynamic systems theory approach to second language acquisition. Bilingualism, Language and Cognition, 10(1), 7\u201321. https:\/\/doi.org\/10.1017\/S1366728906002732","journal-title":"Bilingualism, Language and Cognition"},{"key":"445_CR16","doi-asserted-by":"crossref","unstructured":"Derkach, N. G., Parker, P. C. (2025). Utilizing control-value theory to test relationships between cognitive appraisals, boredom, and perceived success in competitive sport practices. Journal of Sport & Exercise Psychology, 47(1).","DOI":"10.1080\/24711616.2025.2600350"},{"key":"445_CR17","doi-asserted-by":"publisher","unstructured":"Dinh, T. H., Nguyen V. H., Diep, N. N., Ekram, H., & Dusit, N. (2023). Markov decision process and reinforcement learning, in Deep Reinforcement Learning for Wireless Communications and Networking: Theory, Applications and Implementation, IEEE, pp.25\u201336. https:\/\/doi.org\/10.1002\/9781119873747.ch2.","DOI":"10.1002\/9781119873747.ch2"},{"key":"445_CR18","doi-asserted-by":"publisher","first-page":"568","DOI":"10.1007\/s40593-019-00187-x","volume":"29","author":"S Doroudi","year":"2019","unstructured":"Doroudi, S., Aleven, V., & Brunskill, E. (2019). Where\u2019s the reward? International Journal of Artificial Intelligence in Education, 29, 568\u2013620. https:\/\/doi.org\/10.1007\/s40593-019-00187-x","journal-title":"International Journal of Artificial Intelligence in Education"},{"key":"445_CR19","doi-asserted-by":"publisher","first-page":"75","DOI":"10.1016\/j.tele.2019.01.006","volume":"39","author":"J Fang","year":"2019","unstructured":"Fang, J., Tang, L., Yang, J., & Peng, M. (2019). Social interaction in MOOCs: The mediating effects of immersive experience and psychological needs satisfaction. Telematics and Informatics, 39, 75\u201391. https:\/\/doi.org\/10.1016\/j.tele.2019.01.006","journal-title":"Telematics and Informatics"},{"issue":"10","key":"445_CR20","doi-asserted-by":"publisher","first-page":"906","DOI":"10.1037\/0003-066X.34.10.906","volume":"34","author":"JH Flavell","year":"1979","unstructured":"Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive\u2013developmental inquiry. American Psychologist, 34(10), 906.","journal-title":"American Psychologist"},{"issue":"1","key":"445_CR21","doi-asserted-by":"publisher","first-page":"59","DOI":"10.3102\/00346543074001059","volume":"74","author":"JA Fredricks","year":"2004","unstructured":"Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74(1), 59\u2013109. https:\/\/doi.org\/10.3102\/00346543074001059","journal-title":"Review of Educational Research"},{"key":"445_CR22","doi-asserted-by":"publisher","first-page":"763","DOI":"10.1007\/978-1-4614-2018-7_37","volume-title":"Handbook of Research on Student Engagement","author":"JA Fredricks","year":"2012","unstructured":"Fredricks, J. A., & McColskey, W. (2012). The measurement of student engagement: A comparative analysis of various methods and student self-report instruments. Handbook of Research on Student Engagement (pp. 763\u2013782). Springer."},{"issue":"1","key":"445_CR23","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-024-79776-3","volume":"14","author":"P Fu","year":"2024","unstructured":"Fu, P., Gao, C., Chen, X., Zhang, Z., Chen, J., & Yang, D. (2024). Proactive personality and its impact on online learning engagement through positive emotions and learning motivation. Scientific Reports, 14(1), Article 28144. https:\/\/doi.org\/10.1038\/s41598-024-79776-3","journal-title":"Scientific Reports"},{"key":"445_CR24","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2023.104795","volume":"200","author":"C Gherghel","year":"2023","unstructured":"Gherghel, C., Yasuda, S., & Kita, Y. (2023). Interaction during online classes fosters engagement with learning and self-directed study both in the first and second years of the COVID-19 pandemic. Computers & Education, 200, Article 104795.","journal-title":"Computers & Education"},{"key":"445_CR25","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2025.105422","author":"Z Han","year":"2025","unstructured":"Han, Z., Ying, R., Huang, C., Tsai, C. C., Wang, X., & He, T. (2025). Identifying students\u2019 metacognition patterns by their needs for cognitive closure in human-GenAI collaboration. Computers & Education. https:\/\/doi.org\/10.1016\/j.compedu.2025.105422","journal-title":"Computers & Education"},{"issue":"1","key":"445_CR26","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1080\/09523987.2021.1976824","volume":"59","author":"J Holopainen","year":"2022","unstructured":"Holopainen, J., L\u00e4htev\u00e4noja, A., Lammassaari, H., et al. (2022). Best of both worlds: A model combining online and mixed reality learning environments based on cognitive affordances. Educational Media International, 59(1), 45\u201361. https:\/\/doi.org\/10.1080\/09523987.2021.1976824","journal-title":"Educational Media International"},{"key":"445_CR27","volume-title":"Dynamic Programming and Markov Processes","author":"RA Howard","year":"1960","unstructured":"Howard, R. A. (1960). Dynamic Programming and Markov Processes. John Wiley."},{"key":"445_CR28","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2025.1542652","volume":"16","author":"J Hu","year":"2025","unstructured":"Hu, J., & Xiao, W. (2025). What are the influencing factors of online learning engagement? A systematic literature review. Frontiers in Psychology, 16, Article 1542652. https:\/\/doi.org\/10.3389\/fpsyg.2025.1542652","journal-title":"Frontiers in Psychology"},{"issue":"1","key":"445_CR29","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1080\/10705519909540118","volume":"6","author":"LT Hu","year":"1999","unstructured":"Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1\u201355. https:\/\/doi.org\/10.1080\/10705519909540118","journal-title":"Structural Equation Modeling: A Multidisciplinary Journal"},{"key":"445_CR30","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2019.05.015","volume":"140","author":"CQ Huang","year":"2019","unstructured":"Huang, C. Q., Han, Z. M., Li, M. X., Jong, M. S. Y., & Tsai, C. C. (2019). Investigating students\u2019 interaction patterns and dynamic learning sentiments in online discussions. Computers & Education, 140, Article 103589. https:\/\/doi.org\/10.1016\/j.compedu.2019.05.015","journal-title":"Computers & Education"},{"issue":"6","key":"445_CR31","doi-asserted-by":"publisher","first-page":"18","DOI":"10.14742\/ajet.6623","volume":"37","author":"HW Huang","year":"2021","unstructured":"Huang, H. W. (2021). Effects of smartphone-based collaborative vlog projects on EFL learners\u2019 speaking performance and learning engagement. Australasian Journal of Educational Technology, 37(6), 18\u201340. https:\/\/doi.org\/10.14742\/ajet.6623","journal-title":"Australasian Journal of Educational Technology"},{"issue":"1","key":"445_CR32","doi-asserted-by":"publisher","first-page":"104","DOI":"10.3991\/ijep.v12i1.25295","volume":"12","author":"H Ismail","year":"2022","unstructured":"Ismail, H., Khelifi, A., & Harous, S. (2022). A cognitive style based framework for usability evaluation of online lecturing platforms-A case study on Zoom and Teams. International Journal of Engineering Pedagogy, 12(1), 104\u2013122. https:\/\/doi.org\/10.3991\/ijep.v12i1.25295","journal-title":"International Journal of Engineering Pedagogy"},{"key":"445_CR33","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2021.104271","volume":"173","author":"LX Jensen","year":"2021","unstructured":"Jensen, L. X., Bearman, M., & Boud, D. (2021). Understanding feedback in online learning\u2013A critical review and metaphor analysis. Computers & Education, 173, Article 104271. https:\/\/doi.org\/10.1016\/j.compedu.2021.104271","journal-title":"Computers & Education"},{"key":"445_CR34","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1016\/j.compedu.2018.02.013","volume":"122","author":"Y Jung","year":"2018","unstructured":"Jung, Y., & Lee, J. (2018). Learning engagement and persistence in massive open online courses (MOOCS). Computers & Education, 122, 9\u201322. https:\/\/doi.org\/10.1016\/j.compedu.2018.02.013","journal-title":"Computers & Education"},{"key":"445_CR35","doi-asserted-by":"publisher","unstructured":"Khosravifar, B., Bouchet, F., Feyzi-Behnagh, R., Azevedo, R., & Harley, J. M. (2013). Using intelligent multi-agent systems to model and foster self-regulated learning: a theoretically-based approach using Markov decision process. In 2013 IEEE 27th International Conference on Advanced Information Networking and Applications, 413\u2013420. IEEE. https:\/\/doi.org\/10.1109\/AINA.2013.70.","DOI":"10.1109\/AINA.2013.70"},{"key":"445_CR36","doi-asserted-by":"publisher","DOI":"10.1080\/2331186X.2025.2486560","author":"Y Lao","year":"2025","unstructured":"Lao, Y., Wang, P., Liao, S., & Li, C. (2025). Characteristics of self-efficacy of high school students and its relationship with test performance in mathematics subject. Cogent Education. https:\/\/doi.org\/10.1080\/2331186X.2025.2486560","journal-title":"Cogent Education"},{"key":"445_CR37","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1075\/lllt.48.02lar","volume-title":"Complexity theory and language development: In celebration of Diane Larsen-Freeman","author":"D Larsen-Freeman","year":"2017","unstructured":"Larsen-Freeman, D. (2017). Chapter 1. Complexity theory: The lessons continue. Complexity theory and language development: In celebration of Diane Larsen-Freeman (pp. 11\u201350). John Benjamins Publishing Company."},{"issue":"1","key":"445_CR38","doi-asserted-by":"publisher","first-page":"317","DOI":"10.1111\/modl.12693","volume":"105","author":"C Li","year":"2021","unstructured":"Li, C. (2021). A control\u2013value theory approach to boredom in English classes among university students in China. The Modern Language Journal, 105(1), 317\u2013334. https:\/\/doi.org\/10.1111\/modl.12693","journal-title":"The Modern Language Journal"},{"issue":"1","key":"445_CR39","doi-asserted-by":"publisher","first-page":"146","DOI":"10.3991\/ijet.v17i01.28715","volume":"17","author":"M Li","year":"2022","unstructured":"Li, M. (2022). Learning behaviors and cognitive participation in online-offline hybrid learning environment. International Journal of Emerging Technologies in Learning, 17(1), 146\u2013159. https:\/\/doi.org\/10.3991\/ijet.v17i01.28715","journal-title":"International Journal of Emerging Technologies in Learning"},{"issue":"2","key":"445_CR40","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1080\/17501229.2023.2240316","volume":"18","author":"G Liu","year":"2024","unstructured":"Liu, G., & Ma, C. (2024). Measuring EFL learners\u2019 use of ChatGPT in informal digital learning of English based on the technology acceptance model. Innovation in Language Learning and Teaching, 18(2), 125\u2013138. https:\/\/doi.org\/10.1080\/17501229.2023.2240316","journal-title":"Innovation in Language Learning and Teaching"},{"issue":"5","key":"445_CR41","doi-asserted-by":"publisher","first-page":"4292","DOI":"10.1109\/LRA.2025.3551098","volume":"10","author":"J Liu","year":"2025","unstructured":"Liu, J., Xu, C., Hang, P., Sun, J., Ding, M., Zhan, W., & Tomizuka, M. (2025). Language-driven policy distillation for cooperative driving in multi-agent reinforcement learning. IEEE Robotics and Automation Letters, 10(5), 4292\u20134299. https:\/\/doi.org\/10.1109\/LRA.2025.3551098","journal-title":"IEEE Robotics and Automation Letters"},{"key":"445_CR42","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2022.104461","volume":"181","author":"S Liu","year":"2022","unstructured":"Liu, S., Liu, S., Liu, Z., Peng, X., & Yang, Z. (2022). Automated detection of emotional and cognitive engagement in MOOC discussions to predict learning achievement. Computers & Education, 181, Article 104461. https:\/\/doi.org\/10.1016\/j.compedu.2022.104461","journal-title":"Computers & Education"},{"key":"445_CR43","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2024.105109","volume":"219","author":"Z Liu","year":"2024","unstructured":"Liu, Z., Tang, Q., Ouyang, F., Long, T., & Liu, S. (2024). Profiling students\u2019 learning engagement in MOOC discussions to identify learning achievement: An automated configurational approach. Computers & Education, 219, Article 105109. https:\/\/doi.org\/10.1016\/j.compedu.2024.105109","journal-title":"Computers & Education"},{"key":"445_CR44","doi-asserted-by":"publisher","DOI":"10.1016\/j.learninstruc.2018.08.002","volume":"70","author":"K Loderer","year":"2020","unstructured":"Loderer, K., Pekrun, R., & Lester, J. C. (2020). Beyond cold technology: A systematic review and meta-analysis on emotions in technology-based learning environments. Learning and Instruction, 70, Article 101162. https:\/\/doi.org\/10.1016\/j.learninstruc.2018.08.002","journal-title":"Learning and Instruction"},{"key":"445_CR45","doi-asserted-by":"publisher","DOI":"10.1007\/s11423-025-10533-7","author":"K Lu","year":"2025","unstructured":"Lu, K., Ji, T., Lu, F., et al. (2025). Bridging the digital divide: The mediating role of learning engagement between technology usage approaches and higher order thinking skills in a technology-enhanced inquiry-based learning environment. Educational Technology Research and Development. https:\/\/doi.org\/10.1007\/s11423-025-10533-7","journal-title":"Educational Technology Research and Development"},{"key":"445_CR46","doi-asserted-by":"publisher","DOI":"10.1016\/j.system.2025.103689","volume":"131","author":"L Luan","year":"2025","unstructured":"Luan, L., Jing, B., Hong, J. C., & Lin, P. H. (2025). The mediating effects of online learning engagement on the relationship between Chinese university students\u2019 L2 grit and their English language achievement. System, 131, Article 103689. https:\/\/doi.org\/10.1016\/j.system.2025.103689","journal-title":"System"},{"issue":"7","key":"445_CR47","doi-asserted-by":"publisher","first-page":"1509","DOI":"10.1080\/02699931.2020.1760213","volume":"34","author":"M Mailliez","year":"2020","unstructured":"Mailliez, M., Bollon, T., Graton, A., & Hot, P. (2020). Can the induction of incidental positive emotions lead to different performances in sequential decision-making? Cognition and Emotion, 34(7), 1509\u20131516. https:\/\/doi.org\/10.1080\/02699931.2020.1760213","journal-title":"Cognition and Emotion"},{"issue":"3","key":"445_CR48","doi-asserted-by":"publisher","first-page":"270","DOI":"10.1017\/S1355617721000266","volume":"28","author":"AO Massarwe","year":"2022","unstructured":"Massarwe, A. O., Nissan, N., & Gabay, Y. (2022). Atypical reinforcement learning in developmental dyslexia. Journal of the International Neuropsychological Society, 28(3), 270\u2013280. https:\/\/doi.org\/10.1017\/S1355617721000266","journal-title":"Journal of the International Neuropsychological Society"},{"key":"445_CR49","unstructured":"Ng, A. Y., Harada, D., & Russell, S. (1999). Policy invariance under reward transformations: Theory and application to reward shaping. Proceedings of the Sixteenth International Conference on Machine Learning, 278\u2013287. https:\/\/ai.stanford.edu\/~ang\/papers\/shaping-icml99.pdf"},{"issue":"6","key":"445_CR50","doi-asserted-by":"publisher","first-page":"1","DOI":"10.2224\/sbp.9118","volume":"48","author":"X Pan","year":"2020","unstructured":"Pan, X., & Shao, H. (2020). Teacher online feedback and learning motivation: Learning engagement as a mediator. Social Behavior and Personality: An International Journal, 48(6), 1\u201310. https:\/\/doi.org\/10.2224\/sbp.9118","journal-title":"Social Behavior and Personality: An International Journal"},{"key":"445_CR51","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijer.2021.101772","volume":"108","author":"PC Parker","year":"2021","unstructured":"Parker, P. C., Perry, R. P., Hamm, J. M., et al. (2021). A motivation perspective on achievement appraisals, emotions, and performance in an online learning environment. International Journal of Educational Research, 108, Article 101772. https:\/\/doi.org\/10.1016\/j.ijer.2021.101772","journal-title":"International Journal of Educational Research"},{"issue":"4","key":"445_CR52","doi-asserted-by":"publisher","first-page":"315","DOI":"10.1007\/s10648-006-9029-9","volume":"18","author":"R Pekrun","year":"2006","unstructured":"Pekrun, R. (2006). The control-value theory of achievement emotions: Assumptions, corollaries, and implications for educational research and practice. Educational Psychology Review, 18(4), 315\u2013341. https:\/\/doi.org\/10.1007\/s10648-006-9029-9","journal-title":"Educational Psychology Review"},{"key":"445_CR53","volume-title":"Sequential Decision Problems in Online Education","author":"C Pike-Burke","year":"2019","unstructured":"Pike-Burke, C. (2019). Sequential Decision Problems in Online Education. Lancaster University."},{"issue":"2","key":"445_CR54","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1177\/10283153211042091","volume":"27","author":"I Poort","year":"2023","unstructured":"Poort, I., Jansen, E., & Hofman, A. (2023). Cultural intelligence and openness to experiences pave the way for cognitive engagement in intercultural group work. Journal of Studies in International Education, 27(2), 277\u2013297. https:\/\/doi.org\/10.1177\/10283153211042091","journal-title":"Journal of Studies in International Education"},{"issue":"6","key":"445_CR55","doi-asserted-by":"publisher","first-page":"1339","DOI":"10.3102\/0002831218786689","volume":"55","author":"DW Putwain","year":"2018","unstructured":"Putwain, D. W., Pekrun, R., Nicholson, L. J., Symes, W., Becker, S., & Marsh, H. W. (2018). Control-value appraisals, enjoyment, and boredom in mathematics: A longitudinal latent interaction analysis. American Educational Research Journal, 55(6), 1339\u20131368. https:\/\/doi.org\/10.3102\/0002831218786689","journal-title":"American Educational Research Journal"},{"issue":"1","key":"445_CR56","doi-asserted-by":"publisher","first-page":"125","DOI":"10.18860\/ijazarabi.V8i1.26476","volume":"8","author":"R Raswan","year":"2025","unstructured":"Raswan, R., Wahab, M. A., Qodri, M., Mustar, M., & Busri, H. (2025). Students\u2019 satisfaction and self-efficacy in Arabic teacher education using SPACE and Zoom platforms in education 5.0. Ijaz Arabi Journal of Arabic Learning, 8(1), 125\u2013136. https:\/\/doi.org\/10.18860\/ijazarabi.V8i1.26476","journal-title":"Ijaz Arabi Journal of Arabic Learning"},{"issue":"1","key":"445_CR57","doi-asserted-by":"publisher","first-page":"43","DOI":"10.17398\/1695-288x.23.1.43","volume":"23","author":"T Ribeirinha","year":"2024","unstructured":"Ribeirinha, T., & Silva, B. (2024). Student engagement in the flipped classroom model implemented in online learning. Revista Latinoamericana De Tecnolog\u00eda Educativa, 23(1), 43\u201359. https:\/\/doi.org\/10.17398\/1695-288x.23.1.43","journal-title":"Revista Latinoamericana De Tecnolog\u00eda Educativa"},{"key":"445_CR58","doi-asserted-by":"publisher","DOI":"10.1016\/j.compedu.2025.105386","author":"R Schiller","year":"2025","unstructured":"Schiller, R., Fleckenstein, J., H\u00f6ft, L., Horbach, A., & Meyer, J. (2025). On the role of engagement in automated feedback effectiveness: Insights from keystroke logging. Computers & Education. https:\/\/doi.org\/10.1016\/j.compedu.2025.105386","journal-title":"Computers & Education"},{"issue":"1","key":"445_CR59","doi-asserted-by":"publisher","DOI":"10.1186\/s40359-025-02967-8","volume":"13","author":"Y Shao","year":"2025","unstructured":"Shao, Y., Wu, J., Li, Y., Lu, Q., & Wang, Z. (2025). The impact of digital technology use on EFL students\u2019 English academic performance: The mediating roles of emotional intelligence and learning engagement. BMC Psychology, 13(1), Article 638. https:\/\/doi.org\/10.1186\/s40359-025-02967-8","journal-title":"BMC Psychology"},{"key":"445_CR60","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1016\/j.iheduc.2013.04.001","volume":"19","author":"D Shen","year":"2013","unstructured":"Shen, D., Cho, M. H., Tsai, C. L., & Marra, R. (2013). Unpacking online learning experiences: Online learning self-efficacy and learning satisfaction. The Internet and Higher Education, 19, 10\u201317. https:\/\/doi.org\/10.1016\/j.iheduc.2013.04.001","journal-title":"The Internet and Higher Education"},{"key":"445_CR61","doi-asserted-by":"publisher","unstructured":"\u0160trupl, M., Szehr, O., Faccio, F., Ashley, D. R., Srivastava, R. K., & Schmidhuber, J. (2025). On the convergence and stability of upside-down reinforcement learning, goal-conditioned supervised learning, and online decision transformers. arXiv preprint arXiv:2502.05672. https:\/\/doi.org\/10.48550\/arXiv.2502.05672","DOI":"10.48550\/arXiv.2502.05672"},{"key":"445_CR62","doi-asserted-by":"publisher","DOI":"10.1201\/b18188","volume-title":"Statistical Reinforcement Learning: Modern Machine Learning Approaches","author":"M Sugiyama","year":"2015","unstructured":"Sugiyama, M. (2015). Statistical Reinforcement Learning: Modern Machine Learning Approaches. CRC Press."},{"key":"445_CR63","volume-title":"Reinforcement Learning: An Introduction","author":"RS Sutton","year":"2018","unstructured":"Sutton, R. S., & Barto, A. G. (2018). Reinforcement Learning: An Introduction (2nd ed.). MIT press.","edition":"2"},{"key":"445_CR64","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2017.00312","volume":"8","author":"EM Tartaglia","year":"2017","unstructured":"Tartaglia, E. M., Clarke, A. M., & Herzog, M. H. (2017). What to choose next? A paradigm for testing human sequential decision making. Frontiers in Psychology, 8, Article 312. https:\/\/doi.org\/10.3389\/fpsyg.2017.00312","journal-title":"Frontiers in Psychology"},{"key":"445_CR65","doi-asserted-by":"publisher","unstructured":"Tokic, M. (2010). Adaptive \u03b5-greedy exploration in reinforcement learning based on value differences. In Dillmann, R., Beyerer, J., Hanebeck, U.D., Schultz, T. (eds) KI 2010: Advances in Artificial Intelligence. Lecture Notes in Computer Science, Springer, Berlin, Heidelberg. 203\u2013210. https:\/\/doi.org\/10.1007\/978-3-642-16111-7_23.","DOI":"10.1007\/978-3-642-16111-7_23"},{"issue":"4","key":"445_CR66","doi-asserted-by":"publisher","first-page":"472","DOI":"10.1080\/01587919.2020.1821604","volume":"41","author":"CL Tsai","year":"2020","unstructured":"Tsai, C. L., Cho, M. H., Marra, R., & Shen, D. (2020). The self-efficacy questionnaire for online learning (SeQoL). Distance Education, 41(4), 472\u2013489. https:\/\/doi.org\/10.1080\/01587919.2020.1821604","journal-title":"Distance Education"},{"key":"445_CR67","doi-asserted-by":"publisher","DOI":"10.1016\/j.lindif.2022.102128","volume":"94","author":"Y Wang","year":"2022","unstructured":"Wang, Y., Cao, Y., Gong, S., Wang, Z., Li, N., & Ai, L. (2022). Interaction and learning engagement in online learning: The mediating roles of online learning self-efficacy and academic emotions. Learning and Individual Differences, 94, Article 102128. https:\/\/doi.org\/10.1016\/j.lindif.2022.102128","journal-title":"Learning and Individual Differences"},{"key":"445_CR68","doi-asserted-by":"publisher","DOI":"10.1016\/j.system.2025.103753","volume":"133","author":"Y Wang","year":"2025","unstructured":"Wang, Y., Wang, X., & Yao, X. (2025). A polynomial regression and surface response analysis of the (in) congruence of enjoyment and anxiety in online EFL learning. System, 133, Article 103753.","journal-title":"System"},{"key":"445_CR69","volume-title":"Reinforcement Learning and Stochastic Optimization: A Unified Framework for Sequential Decisions","author":"BP Warren","year":"2022","unstructured":"Warren, B. P. (2022). Reinforcement Learning and Stochastic Optimization: A Unified Framework for Sequential Decisions. John Wiley & Sons Inc."},{"key":"445_CR70","volume-title":"Learning from Delayed Rewards","author":"CJCH Watkins","year":"1989","unstructured":"Watkins, C. J. C. H. (1989). Learning from Delayed Rewards. Royal Holloway University of London."},{"issue":"3","key":"445_CR71","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1007\/BF00992698","volume":"8","author":"CJ Watkins","year":"1992","unstructured":"Watkins, C. J., & Dayan, P. (1992). Q-learning. Machine Learning, 8(3), 279\u2013292. https:\/\/doi.org\/10.1007\/BF00992698","journal-title":"Machine Learning"},{"issue":"1","key":"445_CR72","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-024-51887-5","volume":"15","author":"TJ Wieczorek","year":"2024","unstructured":"Wieczorek, T. J., Tchumatchenko, T., Wert-Carvajal, C., & Eggl, M. F. (2024). A framework for the emergence and analysis of language in social learning agents. Nature Communications, 15(1), Article 7590. https:\/\/doi.org\/10.1038\/s41467-024-51887-5","journal-title":"Nature Communications"},{"key":"445_CR73","unstructured":"Winne, P. H., & Hadwin, A. F. (1998). Studying as self-regulated learning. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in educational theory and practice, Lawrence Erlbaum Associates Publishers, pp. 277\u2013304."},{"key":"445_CR74","doi-asserted-by":"publisher","unstructured":"Wu, W., Jiang, L., Fang, L., Xu, Y., Zhou, H., & Yu, H. (2025). The relationship between digital learning resource quality, attention control, academic self-efficacy, and college students\u2019 learning engagement: the moderating role of metacognition. Asia Pacific Journal of Education, 1\u201319. https:\/\/doi.org\/10.1080\/02188791.2024.2444294","DOI":"10.1080\/02188791.2024.2444294"},{"key":"445_CR75","doi-asserted-by":"publisher","unstructured":"Yang, L., & Li, R. (2024). Modelling Chinese EFL learners\u2019 engagement in language MOOCs: a control-value perspective. Computer Assisted Language Learning, 1\u201324. https:\/\/doi.org\/10.1080\/09588221.2024.2386543","DOI":"10.1080\/09588221.2024.2386543"},{"key":"445_CR76","doi-asserted-by":"publisher","first-page":"960","DOI":"10.17179\/excli2024-7502","volume":"23","author":"S Yokoyama","year":"2024","unstructured":"Yokoyama, S. (2024). Impact of academic self-efficacy on online learning outcomes: A recent literature review. EXCLI Journal, 23, 960\u2013966. https:\/\/doi.org\/10.17179\/excli2024-7502","journal-title":"EXCLI Journal"},{"issue":"1","key":"445_CR77","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s41239-019-0171-0","volume":"16","author":"O Zawacki-Richter","year":"2019","unstructured":"Zawacki-Richter, O., Mar\u00edn, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education\u2013where are the educators? International Journal of Educational Technology in Higher Education, 16(1), 1\u201327. https:\/\/doi.org\/10.1186\/s41239-019-0171-0","journal-title":"International Journal of Educational Technology in Higher Education"},{"issue":"1","key":"445_CR78","doi-asserted-by":"publisher","DOI":"10.1007\/s10212-024-00901-x","volume":"40","author":"C Zeng","year":"2025","unstructured":"Zeng, C. (2025). Exploring the roles of online learning engagement, achievement goals, and self-efficacy in internet-specific epistemic justification. European Journal of Psychology of Education, 40(1), Article 2. https:\/\/doi.org\/10.1007\/s10212-024-00901-x","journal-title":"European Journal of Psychology of Education"},{"key":"445_CR79","doi-asserted-by":"publisher","unstructured":"Zheng, Q., & Pan, X. (2025). Examining the impact of online teacher feedback on learning engagement: the mediating roles of technological self-efficacy and learning motivation. Interactive Learning Environments, 1\u201314. https:\/\/doi.org\/10.1080\/10494820.2025.2547263","DOI":"10.1080\/10494820.2025.2547263"},{"issue":"6","key":"445_CR80","doi-asserted-by":"publisher","DOI":"10.3390\/educsci14060664","volume":"14","author":"Q Zhou","year":"2024","unstructured":"Zhou, Q., Zhang, H., & Li, F. (2024). The impact of online interactive teaching on university students\u2019 deep learning\u2014The perspective of self-determination. Education Sciences, 14(6), Article 664. https:\/\/doi.org\/10.3390\/educsci14060664","journal-title":"Education Sciences"},{"issue":"11","key":"445_CR81","doi-asserted-by":"publisher","first-page":"13344","DOI":"10.1109\/TPAMI.2023.3292075","volume":"45","author":"Z Zhu","year":"2023","unstructured":"Zhu, Z., Lin, K., Jain, A. K., & Zhou, J. (2023). Transfer learning in deep reinforcement learning: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(11), 13344\u201313362. https:\/\/doi.org\/10.1109\/TPAMI.2023.3292075","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"445_CR82","doi-asserted-by":"publisher","unstructured":"Zimmerman, B. J. (2000). Attaining self-regulation: A social cognitive perspective. In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of Self-regulation, Academic Press, pp. 13\u201339. https:\/\/doi.org\/10.1016\/B978-012109890-2\/50031-7","DOI":"10.1016\/B978-012109890-2\/50031-7"}],"container-title":["Smart Learning Environments"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40561-026-00445-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s40561-026-00445-1","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s40561-026-00445-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,18]],"date-time":"2026-03-18T09:17:13Z","timestamp":1773825433000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1186\/s40561-026-00445-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,18]]},"references-count":82,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,12]]}},"alternative-id":["445"],"URL":"https:\/\/doi.org\/10.1186\/s40561-026-00445-1","relation":{},"ISSN":["2196-7091"],"issn-type":[{"value":"2196-7091","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,18]]},"assertion":[{"value":"25 September 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 February 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 March 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"All procedures performed in the study involving human participants were in accordance with the ethical standards of the institutional and national research ethics committees and with the Declaration of Helsinki (1964) and its later amendments or comparable ethical standards. All experimental procedures were approved by the Academic Ethics Committee of Shandong University. Informed consent was obtained from all participants included in the study.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"During the preparation of this work, the author used ChatGPT and Doubao in order to improve the readability of the manuscript. After using these tools, the author reviewed and edited the content as needed and take full responsibility for the content of the published article.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"No potential conflict of interest was reported by the author.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"21"}}