{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,23]],"date-time":"2026-03-23T10:52:34Z","timestamp":1774263154208,"version":"3.50.1"},"reference-count":47,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2021,4,30]],"date-time":"2021-04-30T00:00:00Z","timestamp":1619740800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Digital library as one of the most important ways in helping students acquire professional knowledge and improve their professional level has gained great attention in recent years. However, its large collection (especially the book resources) hinders students from finding the resources that they are interested in. To overcome this challenge, many researchers have already turned to recommendation algorithms. Compared with traditional recommendation tasks, in the digital library, there are two challenges in book recommendation problems. The first is that users may borrow books that they are not interested in (i.e., noisy borrowing behaviours), such as borrowing books for classmates. The second is that the number of books in a digital library is usually very large, which means one student can only borrow a small set of books in history (i.e., data sparsity issue). As the noisy interactions in students\u2019 borrowing sequences may harm the recommendation performance of a book recommender, we focus on refining recommendations via filtering out data noises. Moreover, due to the the lack of direct supervision information, we treat noise filtering in sequences as a decision-making process and innovatively introduce a reinforcement learning method as our recommendation framework. Furthermore, to overcome the sparsity issue of students\u2019 borrowing behaviours, a clustering-based reinforcement learning algorithm is further developed. Experimental results on two real-world datasets demonstrate the superiority of our proposed method compared with several state-of-the-art recommendation methods.<\/jats:p>","DOI":"10.3390\/info12050198","type":"journal-article","created":{"date-parts":[[2021,4,30]],"date-time":"2021-04-30T10:53:29Z","timestamp":1619780009000},"page":"198","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Exploring Clustering-Based Reinforcement Learning for Personalized Book Recommendation in Digital Library"],"prefix":"10.3390","volume":"12","author":[{"given":"Xinhua","family":"Wang","sequence":"first","affiliation":[{"name":"School of Information Science and Engineering, Shandong Normal University, Jinan 250358, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9014-2034","authenticated-orcid":false,"given":"Yuchen","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Information Science and Engineering, Shandong Normal University, Jinan 250358, China"}]},{"given":"Lei","family":"Guo","sequence":"additional","affiliation":[{"name":"School of Business, Shandong Normal University, Jinan 250358, China"}]},{"given":"Liancheng","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Information Science and Engineering, Shandong Normal University, Jinan 250358, China"}]},{"given":"Baozhong","family":"Gao","sequence":"additional","affiliation":[{"name":"School of Information Science and Engineering, Shandong Normal University, Jinan 250358, China"}]},{"given":"Fangai","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Information Science and Engineering, Shandong Normal University, Jinan 250358, China"}]},{"given":"Wei","family":"Li","sequence":"additional","affiliation":[{"name":"Library, Shandong Normal University, Jinan 250358, China"}]}],"member":"1968","published-online":{"date-parts":[[2021,4,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1145\/138859.138867","article-title":"Using collaborative filtering to weave an information tapestry","volume":"35","author":"Goldberg","year":"1992","journal-title":"Commun. ACM"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Pazzani, M.J., and Billsus, D. (2007). Content-based recommendation systems. The Adaptive Web, Springer.","DOI":"10.1007\/978-3-540-72079-9_10"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"767","DOI":"10.1007\/s11390-011-0176-1","article-title":"Tag-aware recommender systems: A state-of-the-art survey","volume":"26","author":"Zhang","year":"2011","journal-title":"J. Comput. Sci. Technol."},{"key":"ref_4","first-page":"175","article-title":"Knowledge-based recommender systems","volume":"69","author":"Burke","year":"2000","journal-title":"Encycl. Libr. Inf. Syst."},{"key":"ref_5","first-page":"127","article-title":"A model for book inquiry history analysis and book-acquisition recommendation of libraries","volume":"36","author":"Yang","year":"2012","journal-title":"Libr. Collect. Acquis. Tech. Serv."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"396","DOI":"10.1002\/int.21937","article-title":"An OWA-based ranking approach for university books recommendation","volume":"33","author":"Sohail","year":"2018","journal-title":"Int. J. Intell. Syst."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Priyanka, K., Tewari, A.S., and Barman, A.G. (2015, January 23\u201324). Personalised book recommendation system based on opinion mining technique. Proceedings of the 2015 Global Conference on Communication Technologies (GCCT), Thuckalay, India.","DOI":"10.1109\/GCCT.2015.7342668"},{"key":"ref_8","first-page":"435","article-title":"Hierarchical Reinforcement Learning for Course Recommendation in MOOCs","volume":"33","author":"Zhang","year":"2019","journal-title":"AAAI Conf. Artif. Intell."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"363","DOI":"10.1509\/jmkr.37.3.363.18779","article-title":"Internet Recommendation Systems","volume":"37","author":"Ansari","year":"2000","journal-title":"J. Market. Res."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Ziegler, C.N., McNee, S.M., Konstan, J.A., and Lausen, G. (2005, January 10\u201314). Improving recommendation lists through topic diversification. Proceedings of the 14th International Conference on World Wide Web, Chiba, Japan.","DOI":"10.1145\/1060745.1060754"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Konstas, I., Stathopoulos, V., and Jose, J.M. (2009, January 19\u201323). On social networks and collaborative recommendation. Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Boston, MA, USA.","DOI":"10.1145\/1571941.1571977"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1109\/MS.2009.161","article-title":"Recommendation systems for software engineering","volume":"27","author":"Robillard","year":"2009","journal-title":"IEEE Softw."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Smyth, B. (2007). Case-based recommendation. The Adaptive Web, Springer.","DOI":"10.1007\/978-3-540-72079-9_11"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Fu, X., Budzik, J., and Hammond, K.J. (2000, January 9\u201312). Mining navigation history for recommendation. Proceedings of the 5th International Conference on Intelligent User Interfaces, New Orleans, LA, USA.","DOI":"10.1145\/325737.325796"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Drineas, P., Kerenidis, I., and Raghavan, P. (2002, January 19\u201321). Competitive recommendation systems. Proceedings of the Thiry-Fourth Annual ACM Symposium on Theory of Computing, Montreal, QC, Canada.","DOI":"10.1145\/509907.509922"},{"key":"ref_16","unstructured":"Sabitha, S., and Choudhury, T. (2018). Proposed approach for book recommendation based on user k-NN. Advances in Computer and Computational Sciences, Springer."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Goel, A., Khandelwal, D., Mundhra, J., and Tiwari, R. (2018). Intelligent and integrated book recommendation and best price identifier system using machine learning. Intelligent Engineering Informatics, Springer.","DOI":"10.1007\/978-981-10-7566-7_39"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Mikawa, M., Izumi, S., and Tanaka, K. (2011, January 18\u201321). Book recommendation signage system using silhouette-based gait classification. Proceedings of the 2011 10th International Conference on Machine Learning and Applications and Workshops, Honolulu, HI, USA.","DOI":"10.1109\/ICMLA.2011.43"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Xin, L., Haihong, E., Junde, S., Meina, S., and Junjie, T. (2013, January 13\u201315). Collaborative book recommendation based on readers\u2019 borrowing records. Proceedings of the 2013 International Conference on Advanced Cloud and Big Data, Nanjing, China.","DOI":"10.1109\/CBD.2013.14"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Maneewongvatana, S., and Maneewongvatana, S. (2010, January 26\u201329). A recommendation model for personalized book lists. Proceedings of the 2010 10th International Symposium on Communications and Information Technologies, Tokyo, Japan.","DOI":"10.1109\/ISCIT.2010.5664873"},{"key":"ref_21","first-page":"163","article-title":"An active recommendation approach to improve book-acquisition process","volume":"10","author":"Yang","year":"2012","journal-title":"Int. J. Electron. Bus. Manag."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Tewari, A.S., Kumar, A., and Barman, A.G. (2014, January 21\u201322). Book recommendation system based on combine features of content based filtering, collaborative filtering and association rule mining. Proceedings of the 2014 IEEE International Advance Computing Conference (IACC), Gurgaon, India.","DOI":"10.1109\/IAdCC.2014.6779375"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Sohail, S.S., Siddiqui, J., and Ali, R. (2013, January 22\u201325). Book recommendation system using opinion mining technique. Proceedings of the 2013 International Conference on Advances in Computing, Communications and Informatics (ICACCI), Mysore, India.","DOI":"10.1109\/ICACCI.2013.6637421"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Kanetkar, S., Nayak, A., Swamy, S., and Bhatia, G. (2014, January 1\u20132). Web-based personalized hybrid book recommendation system. Proceedings of the 2014 International Conference on Advances in Engineering & Technology Research (ICAETR-2014), Unnao, India.","DOI":"10.1109\/ICAETR.2014.7012952"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Vaz, P.C., Martins de Matos, D., Martins, B., and Calado, P. (2012, January 10\u201314). Improving a hybrid literary book recommendation system through author ranking. Proceedings of the 12th ACM\/IEEE-CS joint conference on Digital Libraries, Washington, DC, USA.","DOI":"10.1145\/2232817.2232904"},{"key":"ref_26","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv."},{"key":"ref_27","unstructured":"Hausknecht, M., and Stone, P. (2015). Deep recurrent q-learning for partially observable mdps. arXiv."},{"key":"ref_28","unstructured":"Wang, Z., Schaul, T., Hessel, M., Van Hasselt, H., Lanctot, M., and De Freitas, N. (2015). Dueling network architectures for deep reinforcement learning. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12\u201317). Deep reinforcement learning with double q-learning. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_30","unstructured":"Schaul, T., Quan, J., Antonoglou, I., and Silver, D. (2015). Prioritized experience replay. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Hessel, M., Modayil, J., Van Hasselt, H., Schaul, T., Ostrovski, G., Dabney, W., Horgan, D., Piot, B., Azar, M., and Silver, D. (2018, January 2\u20137). Rainbow: Combining improvements in deep reinforcement learning. Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11796"},{"key":"ref_32","unstructured":"Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., and Kavukcuoglu, K. (2016, January 20\u201322). Asynchronous methods for deep reinforcement learning. Proceedings of the International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_33","unstructured":"Vezhnevets, A., Mnih, V., Osindero, S., Graves, A., Vinyals, O., and Agapiou, J. (2016). Strategic attentive writer for learning macro-actions. arXiv."},{"key":"ref_34","unstructured":"Vezhnevets, A.S., Osindero, S., Schaul, T., Heess, N., Jaderberg, M., Silver, D., and Kavukcuoglu, K. (2017, January 6\u201311). Feudal networks for hierarchical reinforcement learning. Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_35","unstructured":"Nachum, O., Gu, S.S., Lee, H., and Levine, S. (2018). Data-efficient hierarchical reinforcement learning. arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Theocharous, G., Thomas, P.S., and Ghavamzadeh, M. (2015, January 25\u201331). Personalized ad recommendation systems for life-time value optimization with guarantees. Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence, Buenos Aires, Argentina.","DOI":"10.1145\/2740908.2741998"},{"key":"ref_37","first-page":"1","article-title":"Exploration in interactive personalized music recommendation: A reinforcement learning approach","volume":"11","author":"Wang","year":"2014","journal-title":"ACM Trans. Multimed. Comput. Commun. Appl. TOMM"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Zheng, G., Zhang, F., Zheng, Z., Xiang, Y., Yuan, N.J., Xie, X., and Li, Z. (2018, January 23\u201327). DRN: A deep reinforcement learning framework for news recommendation. Proceedings of the 2018 World Wide Web Conference, Lyon, France.","DOI":"10.1145\/3178876.3185994"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Wang, L., Zhang, W., He, X., and Zha, H. (2018, January 19\u201323). Supervised reinforcement learning with recurrent neural network for dynamic treatment recommendation. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, UK.","DOI":"10.1145\/3219819.3219961"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Zhao, X., Zhang, L., Ding, Z., Xia, L., Tang, J., and Yin, D. (2018, January 19\u201323). Recommendations with negative feedback via pairwise deep reinforcement learning. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, UK.","DOI":"10.1145\/3219819.3219886"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Zhao, X., Xia, L., Zhang, L., Ding, Z., Yin, D., and Tang, J. (2018, January 2\u20137). Deep reinforcement learning for page-wise recommendations. Proceedings of the 12th ACM Conference on Recommender Systems, Vancouver, BC, Canada.","DOI":"10.1145\/3240323.3240374"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Chen, S.Y., Yu, Y., Da, Q., Tan, J., Huang, H.K., and Tang, H.H. (2018, January 19\u201323). Stabilizing reinforcement learning in dynamic environment with application to online recommendation. Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, London, UK.","DOI":"10.1145\/3219819.3220122"},{"key":"ref_43","unstructured":"Rohde, D., Bonner, S., Dunlop, T., Vasile, F., and Karatzoglou, A. (2018). Recogym: A reinforcement learning environment for the problem of product recommendation in online advertising. arXiv."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"2354","DOI":"10.1109\/TKDE.2018.2831682","article-title":"Nais: Neural attentive item similarity model for recommendation","volume":"30","author":"He","year":"2018","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1109\/MIC.2003.1167344","article-title":"Amazon. com recommendations: Item-to-item collaborative filtering","volume":"7","author":"Linden","year":"2003","journal-title":"IEEE Internet Comput."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Kabbur, S., Ning, X., and Karypis, G. (2013, January 11\u201314). Fism: Factored item similarity models for top-n recommender systems. Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Chicago, IL, USA.","DOI":"10.1145\/2487575.2487589"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"He, X., Deng, K., Wang, X., Li, Y., Zhang, Y., and Wang, M. (2020). LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation. arXiv.","DOI":"10.1145\/3397271.3401063"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/12\/5\/198\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:55:59Z","timestamp":1760162159000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/12\/5\/198"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,30]]},"references-count":47,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2021,5]]}},"alternative-id":["info12050198"],"URL":"https:\/\/doi.org\/10.3390\/info12050198","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,30]]}}}