{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,30]],"date-time":"2026-07-30T14:12:28Z","timestamp":1785420748981,"version":"3.56.0"},"reference-count":34,"publisher":"Springer Science and Business Media LLC","issue":"18","license":[{"start":{"date-parts":[[2023,6,17]],"date-time":"2023-06-17T00:00:00Z","timestamp":1686960000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,6,17]],"date-time":"2023-06-17T00:00:00Z","timestamp":1686960000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100007225","name":"Ministry of Science and Technology","doi-asserted-by":"publisher","award":["111-2410-H-025-039-"],"award-info":[{"award-number":["111-2410-H-025-039-"]}],"id":[{"id":"10.13039\/100007225","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Comput &amp; Applic"],"published-print":{"date-parts":[[2025,6]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Massive open online courses (MOOCs) are open online courses designed on the basis of the teaching progress. Videos and learning exercises are used as learning materials in these courses, which are open to numerous users. However, determining the prerequisite knowledge and learning progress of learners is difficult. On the basis of learners\u2019 online learning trajectory, we designed a set of practice questions for a recommendation system for MOOCs, provided suitable practice questions to students through the LINE chatbot (a type of social media software), and used mobile devices to encourage participation in MOOCs. Reinforcement learning, which involves reward function design and iterative solution improvement, was used to set task goals, including those related to course learning and practice question difficulty. The proposed system encouraged certain learning behaviors among students. Students who used the system exhibited an exercise completion rate of 89.97%, which was higher than that of students who did not use the system (47.23%). The system also increased the students\u2019 overall learning effectiveness. Students who used and did not use the proposed system exhibited average midterm scores of 64.73 and 58.21, respectively. We also collected 227 online questionnaires from students. The results of the questionnaires indicated that 90% of the students were satisfied with the system and hoped to continue using it.<\/jats:p>","DOI":"10.1007\/s00521-023-08686-8","type":"journal-article","created":{"date-parts":[[2023,6,17]],"date-time":"2023-06-17T10:01:57Z","timestamp":1686996117000},"page":"11607-11618","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":29,"title":["Massive open online course recommendation system based on a reinforcement learning algorithm"],"prefix":"10.1007","volume":"37","author":[{"given":"Jian-Wei","family":"Tzeng","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nen-Fu","family":"Huang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"An-Chi","family":"Chuang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ting-Wei","family":"Huang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1249-1655","authenticated-orcid":false,"given":"Hong-Yi","family":"Chang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,6,17]]},"reference":[{"issue":"3","key":"8686_CR1","first-page":"28","volume":"14","author":"M Wang","year":"2011","unstructured":"Wang M, Peng J, Cheng B, Zhou H, Liu J (2011) Knowledge visualization for self-regulated learning. Educ Technol Soc 14(3):28\u201342","journal-title":"Educ Technol Soc"},{"key":"8686_CR2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TLT.2016.2518172","volume":"9","author":"JA Ruiperez-Valiente","year":"2016","unstructured":"Ruiperez-Valiente JA, Merino P, Delgado-Kloos C, Niemann K, Scheffel M, Wolpers M (2016) Analyzing the impact of using optional activities in self-regulated learning. IEEE Trans Learn Technol 9:1\u20131","journal-title":"IEEE Trans Learn Technol"},{"issue":"3","key":"8686_CR3","first-page":"1","volume":"36","author":"WX Zhao","year":"2018","unstructured":"Zhao WX, Zhang W, He Y, Xie X, Wen J-R (2018) Automatically learning topics and difficulty levels of problems in online judge systems. ACM Trans Inf Syst 36(3):1\u201333","journal-title":"ACM Trans Inf Syst"},{"key":"8686_CR4","doi-asserted-by":"crossref","unstructured":"Jing Li ZY (2020) Course recommendations in online education based on collaborative filtering recommendation algorithm, pp 1076\u20132787","DOI":"10.1155\/2020\/6619249"},{"key":"8686_CR5","unstructured":"Micarelli A, Stamper J, Panourgia K (eds) (2016) Springer, Cham, pp 267\u2013272"},{"issue":"7","key":"8686_CR6","doi-asserted-by":"publisher","first-page":"1180","DOI":"10.1016\/j.neucom.2007.11.026","volume":"71","author":"J Peters","year":"2008","unstructured":"Peters J, Schaal S (2008) Natural actor-critic. Neurocomputing 71(7):1180\u20131190","journal-title":"Neurocomputing"},{"key":"8686_CR7","unstructured":"Patel PG, Carver N, Rahimi S (2011) Tuning computer gaming agents using q-learning. In: 2011 federated conference on computer science and information systems (FedCSIS). IEEE, pp 581\u2013 588"},{"issue":"11","key":"8686_CR8","doi-asserted-by":"publisher","first-page":"1238","DOI":"10.1177\/0278364913495721","volume":"32","author":"J Kober","year":"2013","unstructured":"Kober J, Bagnell JA, Peters J (2013) Reinforcement learning in robotics: a survey. Int J Robot Res 32(11):1238\u20131274","journal-title":"Int J Robot Res"},{"key":"8686_CR9","doi-asserted-by":"crossref","unstructured":"Wang P, Fan Y, Xia L, Zhao WX, Niu S, Huang J (2020) Kerl: a knowledge-guided reinforcement learning model for sequential recommendation. In: Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval, ser. SIGIR\u201920. Association for Computing Machinery, New York, pp 209\u2013218","DOI":"10.1145\/3397271.3401134"},{"key":"8686_CR10","doi-asserted-by":"publisher","first-page":"2543","DOI":"10.1007\/s00521-021-05855-5","volume":"34","author":"N Peng","year":"2022","unstructured":"Peng N (2022) Research on the effectiveness of English online learning based on neural network. Neural Comput Appl 34:2543\u20132554","journal-title":"Neural Comput Appl"},{"key":"8686_CR11","doi-asserted-by":"crossref","unstructured":"Taghipour N, Kardan A (2008) A hybrid web recommender system based on q-learning. In: Proceedings of the 2008 ACM symposium on applied computing, pp 1164\u20131168","DOI":"10.1145\/1363686.1363954"},{"issue":"6","key":"8686_CR12","doi-asserted-by":"publisher","first-page":"734","DOI":"10.1109\/TKDE.2005.99","volume":"17","author":"G Adomavicius","year":"2005","unstructured":"Adomavicius G, Tuzhilin A (2005) Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans Knowl Data Eng 17(6):734\u2013749","journal-title":"IEEE Trans Knowl Data Eng"},{"key":"8686_CR13","doi-asserted-by":"crossref","unstructured":"Beel J, Langer S, Genzmehr M, Gipp B, Breitinger C, Nurnberger A (2013) Research paper recommender system evaluation: a quantitative literature survey. In: Proceedings of the international workshop on reproducibility and replication in recommender systems evaluation, ser. RepSys \u201913. Association for Computing Machinery, New York, pp 15\u201322","DOI":"10.1145\/2532508.2532512"},{"key":"8686_CR14","doi-asserted-by":"crossref","unstructured":"Imran K, Abdullah N (2009) Building an e-learning recommender system using vector space model and good learners average rating, pp 194\u2013196","DOI":"10.1109\/ICALT.2009.161"},{"key":"8686_CR15","doi-asserted-by":"publisher","first-page":"14167","DOI":"10.1007\/s00521-021-06065-9","volume":"33","author":"K Rama","year":"2021","unstructured":"Rama K, Kumar P, Bhasker B (2021) Deep autoencoders for feature learning with embeddings for recommendations: a novel recommender system solution. Neural Comput Appl 33:14167\u201314177","journal-title":"Neural Comput Appl"},{"key":"8686_CR16","doi-asserted-by":"publisher","first-page":"10719","DOI":"10.1007\/s00521-020-04844-4","volume":"33","author":"N Sivaramakrishnan","year":"2021","unstructured":"Sivaramakrishnan N, Subramaniyaswamy V, Viloria A, Vijayakumar V, Senthilselvan N (2021) A deep learning-based hybrid model for recommendation generation and ranking. Neural Comput Appl 33:10719\u201310736","journal-title":"Neural Comput Appl"},{"key":"8686_CR17","doi-asserted-by":"crossref","unstructured":"Huang R, Lu R (2018) Research on content-based Mooc recommender model. In: 2018 5th international conference on systems and informatics (ICSAI), pp 676\u2013681","DOI":"10.1109\/ICSAI.2018.8599503"},{"key":"8686_CR18","doi-asserted-by":"crossref","unstructured":"Huang Z, Liu Q, Zhai C, Yin Y, Chen E, Gao W, Hu G (2019) Exploring multi-objective exercise recommendations in online education systems. In: Proceedings of the 28th ACM international conference on information and knowledge management, ser. CIKM \u201919. Association for Computing Machinery, New York, pp 1261\u20131270","DOI":"10.1145\/3357384.3357995"},{"key":"8686_CR19","unstructured":"Nthu mooc (2019) https:\/\/mooc.nthu.edu.tw\/"},{"key":"8686_CR20","doi-asserted-by":"crossref","unstructured":"Chuang AC, Huang NF, Tzeng JW, Lee CA, Huang YX, Huang HH (2021) MOOCERS: Exercise recommender system in MOOCs based on reinforcement learning algorithm. In: 8th International Conference on Soft Computing & Machine Intelligence (ISCMI), Cario, Egypt, 2021, pp 186-190","DOI":"10.1109\/ISCMI53840.2021.9654805"},{"issue":"1","key":"8686_CR21","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1037\/h0057123","volume":"30","author":"TL Kelley","year":"1939","unstructured":"Kelley TL (1939) The selection of upper and lower groups for the validation of test items. J Educ Psychol 30(1):17","journal-title":"J Educ Psychol"},{"key":"8686_CR22","doi-asserted-by":"publisher","first-page":"106481","DOI":"10.1016\/j.knosys.2020.106481","volume":"210","author":"Z Wu","year":"2020","unstructured":"Wu Z, Li M, Tang Y, Liang Q (2020) Exercise recommendation based on knowledge concept prediction. Knowl Based Syst 210:106481","journal-title":"Knowl Based Syst"},{"key":"8686_CR23","unstructured":"LM Corporation. Line messenger. https:\/\/line.me\/en\/"},{"issue":"4","key":"8686_CR24","doi-asserted-by":"publisher","first-page":"286","DOI":"10.1016\/S0019-9958(77)90354-0","volume":"34","author":"IH Witten","year":"1977","unstructured":"Witten IH (1977) An adaptive optimal controller for discrete-time markov environments. Inf Control 34(4):286\u2013295","journal-title":"Inf Control"},{"key":"8686_CR25","volume-title":"Advances in neural information processing systems","author":"V Konda","year":"2000","unstructured":"Konda V, Tsitsiklis J (2000) Actor-critic algorithms. In: Solla S, Leen T, Muller K (eds) Advances in neural information processing systems, vol 12. MIT Press, Boca Raton"},{"key":"8686_CR26","unstructured":"Bellman R (2013) Dynamic programming. Dover Publications."},{"key":"8686_CR27","unstructured":"Mnih V, Kavukcuoglu K, Silver D, Graves A, Antonoglou I, Wierstra D, Riedmiller MA (2013) Playing atari with deep reinforcement learning. CoRR abs\/1312.5602"},{"key":"8686_CR28","doi-asserted-by":"crossref","unstructured":"Rao K, Harris C, Irpan A, Levine S, Ibarz J, Khansari M (2020) Rl-cyclegan: reinforcement learning aware simulation-to-real. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (CVPR)","DOI":"10.1109\/CVPR42600.2020.01117"},{"key":"8686_CR29","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1016\/j.ins.2016.09.024","volume":"374","author":"C Park","year":"2016","unstructured":"Park C, Kim D, Oh J, Yu H (2016) Improving top-k recommendation with truster and trustee relationship in user trust network. Inf Sci 374:100\u2013114","journal-title":"Inf Sci"},{"key":"8686_CR30","unstructured":"Schroeder G, Thiele M, Lehner W (2011) Setting Goals and Choosing Metrics for Recommender System Evaluations, UCERSTI2 Workshop at the 5th ACM Conference on Recommender Systems, Chicago, USA, 23 October 2011"},{"key":"8686_CR31","unstructured":"Wang Y, Wang L, Li Y, He D, Chen W, Liu T-Y (2013) A theoretical analysis of NDCG ranking measures. In: Proceedings of the 26th annual conference on learning theory (COLT 2013), vol 8. Citeseer, p 6"},{"key":"8686_CR32","doi-asserted-by":"publisher","first-page":"100880","DOI":"10.1016\/j.iheduc.2022.100880","volume":"56","author":"X Wei","year":"2023","unstructured":"Wei X, Saab N, Admiraal W (2023) Do learners share the same perceived learning outcomes in MOOCs? Identifying the role of motivation, perceived learning support, learning engagement, and self-regulated learning strategies. Internet Higher Educ 56:100880","journal-title":"Internet Higher Educ"},{"key":"8686_CR33","first-page":"1","volume":"25","author":"DFO Onah","year":"2022","unstructured":"Onah DFO, Pang ELL, Sinclair JE (2022) An investigation of self-regulated learning in a novel MOOC platform. J Comput Higher Educ 25:1\u201334","journal-title":"J Comput Higher Educ"},{"key":"8686_CR34","unstructured":"Nemoto T, Beglar D (2014) Likert-scale questionnaires. In: JALT 2013 conference proceedings, pp 1\u20138"}],"container-title":["Neural Computing and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-023-08686-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00521-023-08686-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-023-08686-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T14:55:27Z","timestamp":1750172127000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00521-023-08686-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,17]]},"references-count":34,"journal-issue":{"issue":"18","published-print":{"date-parts":[[2025,6]]}},"alternative-id":["8686"],"URL":"https:\/\/doi.org\/10.1007\/s00521-023-08686-8","relation":{},"ISSN":["0941-0643","1433-3058"],"issn-type":[{"value":"0941-0643","type":"print"},{"value":"1433-3058","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,17]]},"assertion":[{"value":"17 August 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 May 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 June 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"There is no conflict of interest in this study.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}