{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T14:44:50Z","timestamp":1773931490161,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":32,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,7,25]],"date-time":"2020-07-25T00:00:00Z","timestamp":1595635200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Key R&D Program of China","award":["2019YFA0706401"],"award-info":[{"award-number":["2019YFA0706401"]}]},{"DOI":"10.13039\/501100012659","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61672264, 61632002, 61872166, 61872399, 61902005"],"award-info":[{"award-number":["61672264, 61632002, 61872166, 61872399, 61902005"]}],"id":[{"id":"10.13039\/501100012659","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,7,25]]},"DOI":"10.1145\/3397271.3401170","type":"proceedings-article","created":{"date-parts":[[2020,7,25]],"date-time":"2020-07-25T07:50:08Z","timestamp":1595663408000},"page":"871-880","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":31,"title":["MaHRL"],"prefix":"10.1145","author":[{"given":"Dongyang","family":"Zhao","sequence":"first","affiliation":[{"name":"Peking University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Liang","family":"Zhang","sequence":"additional","affiliation":[{"name":"JD.com, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bo","family":"Zhang","sequence":"additional","affiliation":[{"name":"JD.com, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lizhou","family":"Zheng","sequence":"additional","affiliation":[{"name":"JD.com, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yongjun","family":"Bao","sequence":"additional","affiliation":[{"name":"JD.com, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Weipeng","family":"Yan","sequence":"additional","affiliation":[{"name":"JD.com, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2020,7,25]]},"reference":[{"key":"e_1_3_2_1_1_1","first-page":"1","article-title":"Recent advances in hierarchical reinforcement learning","volume":"13","author":"Barto Andrew G.","year":"2003","unstructured":"Andrew G. Barto and Sridhar Mahadevan . 2003 . Recent advances in hierarchical reinforcement learning . Discrete Event Dynamic Systems , Vol. 13 , 1 -- 2 (2003), 41--77. Andrew G. Barto and Sridhar Mahadevan. 2003. Recent advances in hierarchical reinforcement learning. Discrete Event Dynamic Systems, Vol. 13, 1--2 (2003), 41--77.","journal-title":"Discrete Event Dynamic Systems"},{"key":"e_1_3_2_1_2_1","volume-title":"Proceedings of the 14th. conference on Uncertainty in artificial intelligence. Morgan Kaufmann Publishers Inc., 43--52","author":"Breese John S","year":"1998","unstructured":"John S Breese , David Heckerman , and Carl Kadie . 1998 . Empirical analysis of predictive algorithms for collaborative filtering . In Proceedings of the 14th. conference on Uncertainty in artificial intelligence. Morgan Kaufmann Publishers Inc., 43--52 . John S Breese, David Heckerman, and Carl Kadie. 1998. Empirical analysis of predictive algorithms for collaborative filtering. In Proceedings of the 14th. conference on Uncertainty in artificial intelligence. Morgan Kaufmann Publishers Inc., 43--52."},{"key":"e_1_3_2_1_3_1","volume-title":"Hybrid recommender systems: Survey and experiments. User modeling and user-adapted interaction","author":"Burke Robin","year":"2002","unstructured":"Robin Burke . 2002. Hybrid recommender systems: Survey and experiments. User modeling and user-adapted interaction , Vol. 12 , 4 ( 2002 ), 331--370. Robin Burke. 2002. Hybrid recommender systems: Survey and experiments. User modeling and user-adapted interaction, Vol. 12, 4 (2002), 331--370."},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2988450.2988454"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2959100.2959190"},{"key":"e_1_3_2_1_6_1","volume-title":"Deep reinforcement learning in large discrete action spaces. arXiv preprint arXiv:1512.07679","author":"Dulac-Arnold Gabriel","year":"2015","unstructured":"Gabriel Dulac-Arnold , Richard Evans , Hado van Hasselt , Peter Sunehag , Timothy Lillicrap , Jonathan Hunt , Timothy Mann , TheophaneWeber, Thomas Degris , and Ben Coppin . 2015. Deep reinforcement learning in large discrete action spaces. arXiv preprint arXiv:1512.07679 ( 2015 ). Gabriel Dulac-Arnold, Richard Evans, Hado van Hasselt, Peter Sunehag, Timothy Lillicrap, Jonathan Hunt, Timothy Mann, TheophaneWeber, Thomas Degris, and Ben Coppin. 2015. Deep reinforcement learning in large discrete action spaces. arXiv preprint arXiv:1512.07679 (2015)."},{"key":"e_1_3_2_1_7_1","volume-title":"Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939","author":"Hidasi Bal\u00e1zs","year":"2015","unstructured":"Bal\u00e1zs Hidasi , Alexandros Karatzoglou , Linas Baltrunas , and Domonkos Tikk . 2015. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939 ( 2015 ). Bal\u00e1zs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2015. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939 (2015)."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219846"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/582415.582418"},{"key":"e_1_3_2_1_10_1","volume-title":"Proceedings of the 19th international conference on World Wide Web. ACM, 661--670","author":"Li Lihong","unstructured":"Lihong Li , Wei Chu , John Langford , and Robert E. Schapire . 2010. A contextual-bandit approach to personalized news article recommendation . In Proceedings of the 19th international conference on World Wide Web. ACM, 661--670 . Lihong Li, Wei Chu, John Langford, and Robert E. Schapire. 2010. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World Wide Web. ACM, 661--670."},{"key":"e_1_3_2_1_11_1","volume-title":"Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971","author":"Lillicrap Timothy P","year":"2015","unstructured":"Timothy P Lillicrap , Jonathan J Hunt , Alexander Pritzel , Nicolas Heess , Tom Erez , Yuval Tassa , David Silver , and Daan Wierstra . 2015. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 ( 2015 ). Timothy P Lillicrap, Jonathan J Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2015. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 (2015)."},{"key":"e_1_3_2_1_13_1","volume-title":"com recommendations: Item-to-item collaborative filtering","author":"Linden Greg","year":"2003","unstructured":"Greg Linden , Brent Smith , and Jeremy York . 2003. Amazon. com recommendations: Item-to-item collaborative filtering . IEEE Internet computing, Vol. 7 , 1 ( 2003 ), 76--80. Greg Linden, Brent Smith, and Jeremy York. 2003. Amazon. com recommendations: Item-to-item collaborative filtering. IEEE Internet computing, Vol. 7, 1 (2003), 76--80."},{"key":"e_1_3_2_1_14_1","volume-title":"Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602","author":"Mnih Volodymyr","year":"2013","unstructured":"Volodymyr Mnih , Koray Kavukcuoglu , David Silver , Alex Graves , Ioannis Antonoglou , Daan Wierstra , and Martin Riedmiller . 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 ( 2013 ). Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. 2013. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013)."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/336597.336662"},{"key":"e_1_3_2_1_16_1","unstructured":"Ofir Nachum Shixiang Gu Honglak Lee and Sergey Levine. 2018. Data-efficient hierarchical reinforcement learning. In Advances in neural information processing systems.  Ofir Nachum Shixiang Gu Honglak Lee and Sergey Levine. 2018. Data-efficient hierarchical reinforcement learning. In Advances in neural information processing systems."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2010.127"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/245108.245121"},{"key":"e_1_3_2_1_19_1","volume-title":"Recommender systems handbook","author":"Ricci Francesco","unstructured":"Francesco Ricci , Lior Rokach , and Bracha Shapira . 2011. Introduction to recommender systems handbook . In Recommender systems handbook . Springer , 1--35. Francesco Ricci, Lior Rokach, and Bracha Shapira. 2011. Introduction to recommender systems handbook. In Recommender systems handbook. Springer, 1--35."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0004-3702(99)00052-1"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2020\/487"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148176"},{"key":"e_1_3_2_1_23_1","volume-title":"FeUdal networks for hierarchical reinforcement learning. arXiv preprint arXiv:1703.01161","author":"Vezhnevets Alexander Sasha","year":"2017","unstructured":"Alexander Sasha Vezhnevets , Simon Osindero , Tom Schaul , Nicolas Heess , Max Jaderberg , David Silver , and Koray Kavukcuoglu . 2017. FeUdal networks for hierarchical reinforcement learning. arXiv preprint arXiv:1703.01161 ( 2017 ). Alexander Sasha Vezhnevets, Simon Osindero, Tom Schaul, Nicolas Heess, Max Jaderberg, David Silver, and Koray Kavukcuoglu. 2017. FeUdal networks for hierarchical reinforcement learning. arXiv preprint arXiv:1703.01161 (2017)."},{"key":"e_1_3_2_1_24_1","volume-title":"Thirty-Second AAAI Conference on Artificial Intelligence.","author":"Wang Yu","unstructured":"Yu Wang , Jixing Xu , Aohan Wu , Mantian Li , Yang He , Jinghe Hu , and Weipeng P. Yan . 2018. Telepath: understanding users from a human vision perspective in large-scale recommender systems . In Thirty-Second AAAI Conference on Artificial Intelligence. Yu Wang, Jixing Xu, Aohan Wu, Mantian Li, Yang He, Jinghe Hu, and Weipeng P. Yan. 2018. Telepath: understanding users from a human vision perspective in large-scale recommender systems. In Thirty-Second AAAI Conference on Artificial Intelligence."},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3357384.3357936"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2016.7498326"},{"key":"e_1_3_2_1_27_1","volume-title":"Proceedings of the 5th International Workshop on Big Data, Streams and Heterogeneous Source Mining: Algorithms, Systems, Programming Models and Applications at KDD","volume":"53","author":"Yang Hongxia","year":"2016","unstructured":"Hongxia Yang , Quan Lu , Angus Xianen Qiu , and Chun Han . 2016 . Large scale CVR prediction through dynamic transfer learning of global and local features . In Proceedings of the 5th International Workshop on Big Data, Streams and Heterogeneous Source Mining: Algorithms, Systems, Programming Models and Applications at KDD 2016, Vol. 53 . PMLR, 103--119. Hongxia Yang, Quan Lu, Angus Xianen Qiu, and Chun Han. 2016. Large scale CVR prediction through dynamic transfer learning of global and local features. In Proceedings of the 5th International Workshop on Big Data, Streams and Heterogeneous Source Mining: Algorithms, Systems, Programming Models and Applications at KDD 2016, Vol. 53. PMLR, 103--119."},{"key":"e_1_3_2_1_28_1","first-page":"4","article-title":"Hierarchical reinforcement learning for course recommendation in MOOCs","volume":"5","author":"Zhang Jing","year":"2019","unstructured":"Jing Zhang , Bowen Hao , Bo Chen , Cuiping Li , Hong Chen , and Jimeng Sund . 2019 . Hierarchical reinforcement learning for course recommendation in MOOCs . Psychology , Vol. 5 , 4 .64 (2019), 5--65. Jing Zhang, Bowen Hao, Bo Chen, Cuiping Li, Hong Chen, and Jimeng Sund. 2019. Hierarchical reinforcement learning for course recommendation in MOOCs. Psychology, Vol. 5, 4.64 (2019), 5--65.","journal-title":"Psychology"},{"key":"e_1_3_2_1_29_1","volume-title":"Deep learning based recommender system: A survey and new perspectives. arXiv preprint arXiv:1707.07435","author":"Zhang Shuai","year":"2017","unstructured":"Shuai Zhang , Lina Yao , and Aixin Sun . 2017. Deep learning based recommender system: A survey and new perspectives. arXiv preprint arXiv:1707.07435 ( 2017 ). Shuai Zhang, Lina Yao, and Aixin Sun. 2017. Deep learning based recommender system: A survey and new perspectives. arXiv preprint arXiv:1707.07435 (2017)."},{"key":"e_1_3_2_1_30_1","volume-title":"Deep reinforcement learning for page-wise recommendations. arXiv preprint arXiv:1805.02343","author":"Zhao Xiangyu","year":"2018","unstructured":"Xiangyu Zhao , Long Xia , Liang Zhang , Zhuoye Ding , Dawei Yin , and Jiliang Tang . 2018a. Deep reinforcement learning for page-wise recommendations. arXiv preprint arXiv:1805.02343 ( 2018 ). Xiangyu Zhao, Long Xia, Liang Zhang, Zhuoye Ding, Dawei Yin, and Jiliang Tang. 2018a. Deep reinforcement learning for page-wise recommendations. arXiv preprint arXiv:1805.02343 (2018)."},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3219886"},{"key":"e_1_3_2_1_32_1","volume-title":"Deep reinforcement learning for list-wise recommendations. arXiv preprint arXiv:1801.00209","author":"Zhao Xiangyu","year":"2018","unstructured":"Xiangyu Zhao , Liang Zhang , Zhuoye Ding , Dawei Yin , Yihong Zhao , and Jiliang Tang . 2018c. Deep reinforcement learning for list-wise recommendations. arXiv preprint arXiv:1801.00209 ( 2018 ). Xiangyu Zhao, Liang Zhang, Zhuoye Ding, Dawei Yin, Yihong Zhao, and Jiliang Tang. 2018c. Deep reinforcement learning for list-wise recommendations. arXiv preprint arXiv:1801.00209 (2018)."},{"key":"e_1_3_2_1_33_1","volume-title":"Deep interest evolution network for click-through rate prediction. arXiv preprint arXiv:1809.03672","author":"Zhou Guorui","year":"2018","unstructured":"Guorui Zhou , Na Mou , Ying Fan , Qi Pi , Weijie Bian , Chang Zhou , Xiaoqiang Zhu , and Kun Gai . 2018. Deep interest evolution network for click-through rate prediction. arXiv preprint arXiv:1809.03672 ( 2018 ). Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2018. Deep interest evolution network for click-through rate prediction. arXiv preprint arXiv:1809.03672 (2018)."}],"event":{"name":"SIGIR '20: The 43rd International ACM SIGIR conference on research and development in Information Retrieval","location":"Virtual Event China","acronym":"SIGIR '20","sponsor":["SIGIR ACM Special Interest Group on Information Retrieval"]},"container-title":["Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3397271.3401170","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3397271.3401170","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:41:43Z","timestamp":1750200103000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3397271.3401170"}},"subtitle":["Multi-goals Abstraction Based Deep Hierarchical Reinforcement Learning for Recommendations"],"short-title":[],"issued":{"date-parts":[[2020,7,25]]},"references-count":32,"alternative-id":["10.1145\/3397271.3401170","10.1145\/3397271"],"URL":"https:\/\/doi.org\/10.1145\/3397271.3401170","relation":{},"subject":[],"published":{"date-parts":[[2020,7,25]]},"assertion":[{"value":"2020-07-25","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}