{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,23]],"date-time":"2026-02-23T05:55:23Z","timestamp":1771826123835,"version":"3.50.1"},"reference-count":41,"publisher":"Springer Science and Business Media LLC","issue":"6","license":[{"start":{"date-parts":[[2023,6,29]],"date-time":"2023-06-29T00:00:00Z","timestamp":1687996800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,6,29]],"date-time":"2023-06-29T00:00:00Z","timestamp":1687996800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Key and Technology in Henan","award":["222102210048"],"award-info":[{"award-number":["222102210048"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Complex Intell. Syst."],"published-print":{"date-parts":[[2023,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Online recommendation systems process large amounts of information to make personalized recommendations. There has been some progress in research on incorporating knowledge graphs in reinforcement learning for recommendation; however, some challenges still remain. First, in these approaches, an agent cannot switch paths intelligently, because of which, the agent cannot cope with multi-entities and multi-relations in knowledge graphs. Second, these methods do not have predefined targets and thus cannot discover items that are closely related to user-interacted items and latent rich semantic relationships. Third, contemporary methods do not consider long rational paths in knowledge graphs. To address these problems, we propose a deep knowledge reinforcement learning (DKRL) framework, in which path-guided intelligent switching was implemented over knowledge graphs incorporating reinforcement learning; this model integrates predefined target and long logic paths over knowledge graphs for recommendation systems. Specifically, the designed novel path-based intelligent switching algorithm with predefined target enables an agent to switch paths intelligently among multi-entities and multi-relations over knowledge graphs. In addition, the weight of each path is calculated, and the agent switches paths between multiple entities according to path weights. Furthermore, the long logic path has better recommendation performance and interpretability. Extensive experiments with actual data demonstrate that our work improves upon existing methods.The experimental results indicated that DKRL improved the baselines of NDCG@10 by 3.7%, 9.3%, and 4.7%; of HR@10 by 12.39%, 20.8%, and 13.86%; of Prec@10 by 5.17%, 3.57%, 6.2%; of Recall@10 by 3.01%, 4.2%, and 3.37%. The DKRL model achieved more effective recommendation performance using several large benchmark data sets compared with other advanced methods.<\/jats:p>","DOI":"10.1007\/s40747-023-01124-1","type":"journal-article","created":{"date-parts":[[2023,6,29]],"date-time":"2023-06-29T12:02:16Z","timestamp":1688040136000},"page":"7305-7319","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Path-guided intelligent switching over knowledge graphs with deep reinforcement learning for recommendation"],"prefix":"10.1007","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8000-4559","authenticated-orcid":false,"given":"Shaohua","family":"Tao","sequence":"first","affiliation":[]},{"given":"Runhe","family":"Qiu","sequence":"additional","affiliation":[]},{"given":"Yan","family":"Cao","sequence":"additional","affiliation":[]},{"given":"Guoqing","family":"Xue","sequence":"additional","affiliation":[]},{"given":"Yuan","family":"Ping","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,6,29]]},"reference":[{"key":"1124_CR1","doi-asserted-by":"crossref","unstructured":"Zhang H-W, Shen WLF-M, He X-N et al (2016) Discrete collaborative filtering. In: Proceedings of the 39th international ACM SIGIR conference, pp 325\u2013334","DOI":"10.1145\/2911451.2911502"},{"key":"1124_CR2","doi-asserted-by":"crossref","unstructured":"Sarwar B, Karypis JKG, Riedl J (2001) Item-based collaborative filtering recommendation algorithms. In: Proceedings of international conference on world wide web, pp 285\u2013295","DOI":"10.1145\/371920.372071"},{"key":"1124_CR3","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1109\/MC.2009.263","volume":"42","author":"KBR Yehuda","year":"2009","unstructured":"Yehuda KBR, Chris V (2009) Matrix factorization techniques for recommender systems. Computer 42:30\u201337","journal-title":"Computer"},{"key":"1124_CR4","doi-asserted-by":"crossref","unstructured":"He X-N, Zhang M-YKH, Chua T-S (2017) Fast matrix factorization for online recommendation with implicit feedback. In: arXiv:1708.0502","DOI":"10.1145\/2911451.2911489"},{"key":"1124_CR5","doi-asserted-by":"crossref","unstructured":"Zhao Q, YS, Hong L (2017) Gb-cent: Gradient boosted categorical embedding and numerical trees, pp 1311\u20131319","DOI":"10.1145\/3038912.3052668"},{"key":"1124_CR6","doi-asserted-by":"crossref","unstructured":"Wang X, He MW X-N, Feng F-L et al (2019) Neural graph collaborative filtering. In: Proceedings of the 42rd international ACM SIGIR conference on research and development in information retrieval, pp 165\u2013174","DOI":"10.1145\/3331184.3331267"},{"key":"1124_CR7","doi-asserted-by":"crossref","unstructured":"Zhao X-Y, Xia LZL, Ding Z-Y et al (2018) Deep reinforcement learning for page-wise recommendations. arXiv:1805.0234","DOI":"10.1145\/3240323.3240374"},{"key":"1124_CR8","doi-asserted-by":"crossref","unstructured":"Zheng G-J, Zhang Z-HZF-Z, Nicholas Y-X et al (2018) Drn: a deep reinforcement learning framework for news recommendation. In: Proceedings of the 2018 world wide web conference, pp 126\u2013137","DOI":"10.1145\/3178876.3185994"},{"key":"1124_CR9","doi-asserted-by":"crossref","unstructured":"Zou L-X, Xia Z-YDL, Song J-X (2019) Reinforcement learning to optimize long-term user engagement in recommender system. In: Proceedings of the 23th ACM SIGKDD international conference on knowledge discovery and data mining, pp 2810\u20132818","DOI":"10.1145\/3292500.3330668"},{"key":"1124_CR10","doi-asserted-by":"crossref","unstructured":"Xian Y-K, Fu SMZ-H, Melo G-D et al (2019) Reinforcement knowledge graph reasoning for explainable recommendation. In: Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval, pp 285\u2013294","DOI":"10.1145\/3331184.3331203"},{"key":"1124_CR11","doi-asserted-by":"crossref","unstructured":"Wang P-F, Fan LXY, Zhao W-X et al (2020) Kerl: a knowledge-guided reinforcement learning model for sequential recommendation. In: Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval, pp 209\u2013218","DOI":"10.1145\/3397271.3401134"},{"key":"1124_CR12","doi-asserted-by":"crossref","unstructured":"Zhou S-J, Dai H-KCX-Y, Zhang W-N et al (2020) Interactive recommender system via knowledge graph-enhanced reinforcement learning. In: Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval, pp 179\u2013188","DOI":"10.1145\/3397271.3401174"},{"key":"1124_CR13","doi-asserted-by":"crossref","unstructured":"Wang X, Xu X-NHY-K, Cao Y-X et al (2020) Reinforced negative sampling over knowledge graph for recommendation. In: Proceedings of the web conference 2020, pp 99\u2013109","DOI":"10.1145\/3366423.3380098"},{"key":"1124_CR14","doi-asserted-by":"crossref","unstructured":"Li H-Y, Chen C-LLZ-H, Xiao R et al (2021) Path-based deep network for candidate item matching in recommenders. arXiv:2105.0824","DOI":"10.1145\/3404835.3462878"},{"key":"1124_CR15","doi-asserted-by":"crossref","unstructured":"Wang X, Huang D-XWT-L, Liu Z-G (2021) Learning intents behind interactions with knowledge graph for recommendation. In: Proceedings of the international world wide web conference committee 2021","DOI":"10.1145\/3442381.3450133"},{"key":"1124_CR16","doi-asserted-by":"crossref","unstructured":"Xia L-H, Huang YXC, Dai P et al (2020) Knowledge-enhanced hierarchical graph transformer network for multi-behavior recommendation. In: Proceedings of the 34th association for the advancement of artificial intelligence, pp 4486\u20134493","DOI":"10.1609\/aaai.v35i5.16576"},{"key":"1124_CR17","unstructured":"Wang X, Wang C-RXD-X, He X-N (2018) Explainable reasoning over knowledge graphs for recommendation. arXiv:1811.0454"},{"key":"1124_CR18","unstructured":"Hasselt H-V, AG, Silver D (2015) Deep reinforcement learning with double q-learning. arXiv:1509.0646"},{"key":"1124_CR19","doi-asserted-by":"crossref","unstructured":"Deng Z-H, Huang C-DWL, Lai J-H (2019) Deepcf: a unified framework of representation learning and matching function learning in recommender system. arXiv:1901.0470","DOI":"10.1609\/aaai.v33i01.330161"},{"key":"1124_CR20","doi-asserted-by":"crossref","unstructured":"Wang H, Y N-YW, Yeung D (2015) Collaborative deep learning for recommender systems. In: Proceedings of the 19th ACM SIGKDD international conference on knowledge discovery and data mining, pp 1235\u20131244","DOI":"10.1145\/2783258.2783273"},{"key":"1124_CR21","doi-asserted-by":"crossref","unstructured":"Guo H-F, Tang Y-MYR-M, Li Z-G et al (2017) Deepfm: a factorization machine based neural network for ctr prediction. In: IJCAI, pp 1725\u20131731","DOI":"10.24963\/ijcai.2017\/239"},{"key":"1124_CR22","unstructured":"Lu Z-Q, Yang Q (2016) Partially observable Markov decision process for recommender systems. arXiv:1608.07793"},{"key":"1124_CR23","doi-asserted-by":"crossref","unstructured":"Theocharous G, P-ST, Ghavamzadeh M (2015) Personalized ad recommendation systems for life-time value optimization with guarantees. In: IJCAI, pp 1806\u20131812","DOI":"10.1145\/2740908.2741998"},{"key":"1124_CR24","doi-asserted-by":"crossref","unstructured":"Wang X-T, Chen JYY-R, Wu L et al (2018) A reinforcement learning framework for explainable recommendation. In: IEEE international conference on data mining, pp 587\u2013596","DOI":"10.1109\/ICDM.2018.00074"},{"key":"1124_CR25","doi-asserted-by":"crossref","unstructured":"Xiong W-H TH, Wang W-Y (2017) Deeppath: a reinforcement learning method for knowledge graph reasoning. In: Proceedings of the 2017 conference on empirical methods in natural language processing, pp 564\u2013573","DOI":"10.18653\/v1\/D17-1060"},{"key":"1124_CR26","unstructured":"Das R, Dhuliawala MZS, Vilnis L-K et al (2017) Go for a walk and arrive at the answer: reasoning over paths in knowledge bases with reinforcement learning. arXiv:1711.0585"},{"key":"1124_CR27","doi-asserted-by":"crossref","unstructured":"Lin X-V RS, Xiong C (2018) Multi-hop knowledge graph reasoning with reward shaping. arXiv:1808.1056","DOI":"10.18653\/v1\/D18-1362"},{"key":"1124_CR28","unstructured":"Yang B-S, T M (2019) Leveraging knowledge bases in lstms for improving machine reading. arXiv:1902.09091"},{"key":"1124_CR29","doi-asserted-by":"crossref","unstructured":"Tao S-H, Qiu YPR-H, Ma H (2021) Multi-modal knowledge-aware reinforcement learning network for explainable recommendation. Knowl Based Syst 227:107217","DOI":"10.1016\/j.knosys.2021.107217"},{"key":"1124_CR30","doi-asserted-by":"crossref","unstructured":"Cao Y-X, Wang X-NHX, Hu Z-K (2019) Unifying knowledge graph learning and recommendation: towards a better understanding of user preferences. In: Proceedings of the web conference 2019, pp 151\u2013161","DOI":"10.1145\/3308558.3313705"},{"key":"1124_CR31","doi-asserted-by":"crossref","unstructured":"Yang D-Q, Guo Z-YWZ-K, Jian J-Y et al (2018) A knowledge-enhanced deep recommendation framework incorporating gan-based models. In: Proceedings of IEEE international conference on data mining, pp 1368\u20131373","DOI":"10.1109\/ICDM.2018.00187"},{"key":"1124_CR32","doi-asserted-by":"crossref","unstructured":"Wang H-W, Zhang XXF-Z, Guo M-Y (2018) Dkn: deep knowledge-aware network for news recommendation. arXiv:1808.0828","DOI":"10.1145\/3178876.3186175"},{"key":"1124_CR33","doi-asserted-by":"crossref","unstructured":"Huang J, Zhao H-JD W-X, Wen J-R et al (2018) Improving sequential recommendation with knowledge-enhanced memory networks. In: Proceedings of the 41nd international ACM SIGIR conference on research and development in information retrieval, pp 505\u2013514","DOI":"10.1145\/3209978.3210017"},{"key":"1124_CR34","doi-asserted-by":"publisher","first-page":"137","DOI":"10.3390\/a11090137","volume":"11","author":"Q-Y Ai","year":"2018","unstructured":"Ai Q-Y, Azizi XCV, Zhang Y-F (2018) Learning heterogeneous knowledge base embeddings for explainable recommendation, algorithms. Algorithms 11:137\u2013153","journal-title":"Algorithms"},{"key":"1124_CR35","doi-asserted-by":"crossref","unstructured":"Zheng J-Y, J-YM, Wen Y-L (2022) Explainable session-based recommendation with meta-path guided instances and self-attention mechanism. In: Proceedings of the 46nd international ACM SIGIR conference on research and development in information retrieval, pp. 2555\u20132559","DOI":"10.1145\/3477495.3531895"},{"key":"1124_CR36","doi-asserted-by":"crossref","unstructured":"Huang R-r, C-QH, Cui L (2021) Entity-aware collaborative relation network with knowledge graph for recommendation. In: Proceedings of the CIKM 2021, pp 3098\u20133102","DOI":"10.1145\/3459637.3482098"},{"key":"1124_CR37","doi-asserted-by":"crossref","unstructured":"Wang H-W, Zhang XX F-Z, Guo M-Y (2018) Cndbpedia: a never-ending Chinese knowledge extraction system. In: Proceedings of the international conference on industrial, engineering and other applications of applied intelligent systems, pp 428\u2013438","DOI":"10.1007\/978-3-319-60045-1_44"},{"key":"1124_CR38","doi-asserted-by":"crossref","unstructured":"Dong Y-X, N-VC, Swami A (2017) metapath2vec: scalable representation learning for heterogeneous networks. In: Proceedings of the 23th ACM SIGKDD international conference on knowledge discovery and data mining, pp 135\u2013144","DOI":"10.1145\/3097983.3098036"},{"key":"1124_CR39","doi-asserted-by":"publisher","first-page":"529","DOI":"10.1038\/nature14236","volume":"518","author":"V Mnih","year":"2015","unstructured":"Mnih V, Silver KK et al (2015) Human-level control through deep reinforcement learning. Nature 518:529\u2013533","journal-title":"Nature"},{"key":"1124_CR40","doi-asserted-by":"crossref","unstructured":"Rendle S (2012) Factorization machines with libfm. In: ACM transactions on intelligent systems and technology, vol 3, p 57","DOI":"10.1145\/2168752.2168771"},{"key":"1124_CR41","doi-asserted-by":"crossref","unstructured":"Perozzi B, RA-RR, Skiena S (2014) Deepwalk: online learning of social representations. In: Proceedings of the 20th ACM SIGKDD conference on knowledge discovery and data mining, pp 701\u2013710","DOI":"10.1145\/2623330.2623732"}],"container-title":["Complex &amp; Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-023-01124-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40747-023-01124-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-023-01124-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,27]],"date-time":"2023-10-27T19:05:40Z","timestamp":1698433540000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40747-023-01124-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,29]]},"references-count":41,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2023,12]]}},"alternative-id":["1124"],"URL":"https:\/\/doi.org\/10.1007\/s40747-023-01124-1","relation":{},"ISSN":["2199-4536","2198-6053"],"issn-type":[{"value":"2199-4536","type":"print"},{"value":"2198-6053","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,29]]},"assertion":[{"value":"12 December 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 May 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 June 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}