{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,4]],"date-time":"2026-06-04T15:57:52Z","timestamp":1780588672193,"version":"3.54.1"},"reference-count":58,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2022,5,13]],"date-time":"2022-05-13T00:00:00Z","timestamp":1652400000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001691","name":"JSPS KAKENHI","doi-asserted-by":"publisher","award":["JP21H03456"],"award-info":[{"award-number":["JP21H03456"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In this study, we propose a novel music playlist generation method based on a knowledge graph and reinforcement learning. The development of music streaming platforms has transformed the social dynamics of music consumption and paved a new way of accessing and listening to music. The playlist generation is one of the most important multimedia techniques, which aims to recommend music tracks by sensing the vast amount of musical data and the users\u2019 listening histories from music streaming services. Conventional playlist generation methods have difficulty capturing the target users\u2019 long-term preferences. To overcome the difficulty, we use a reinforcement learning algorithm that can consider the target users\u2019 long-term preferences. Furthermore, we introduce the following two new ideas: using the informative knowledge graph data to promote efficient optimization through reinforcement learning, and setting the flexible reward function that target users can design the parameters of itself to guide target users to new types of music tracks. We confirm the effectiveness of the proposed method by verifying the prediction performance based on listening history and the guidance performance to music tracks that can satisfy the target user\u2019s unique preference.<\/jats:p>","DOI":"10.3390\/s22103722","type":"journal-article","created":{"date-parts":[[2022,5,15]],"date-time":"2022-05-15T09:48:22Z","timestamp":1652608102000},"page":"3722","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Controllable Music Playlist Generation Based on Knowledge Graph and Reinforcement Learning"],"prefix":"10.3390","volume":"22","author":[{"given":"Keigo","family":"Sakurai","sequence":"first","affiliation":[{"name":"Graduate School of Information Science and Technology, Hokkaido University, N-14, W-9, Kita-ku, Sapporo 060-0814, Hokkaido, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4474-3995","authenticated-orcid":false,"given":"Ren","family":"Togo","sequence":"additional","affiliation":[{"name":"Faculty of Information Science and Technology, Hokkaido University, N-14, W-9, Kita-ku, Sapporo 060-0814, Hokkaido, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5332-8112","authenticated-orcid":false,"given":"Takahiro","family":"Ogawa","sequence":"additional","affiliation":[{"name":"Faculty of Information Science and Technology, Hokkaido University, N-14, W-9, Kita-ku, Sapporo 060-0814, Hokkaido, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Miki","family":"Haseyama","sequence":"additional","affiliation":[{"name":"Faculty of Information Science and Technology, Hokkaido University, N-14, W-9, Kita-ku, Sapporo 060-0814, Hokkaido, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,5,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"339","DOI":"10.1007\/s12599-015-0401-5","article-title":"Digital transformation strategies","volume":"57","author":"Matt","year":"2015","journal-title":"Bus. Inf. Syst. Eng."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"448","DOI":"10.1109\/TASL.2007.911513","article-title":"A regression approach to music emotion recognition","volume":"16","author":"Yang","year":"2008","journal-title":"IEEE Trans. Audio Speech Lang. Process."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Sakurai, K., Togo, R., Ogawa, T., and Haseyama, M. (2020, January 13\u201316). Music playlist generation based on reinforcement learning using acoustic feature map. Proceedings of the IEEE 9th Global Conference on Consumer Electronics, Kobe, Japan.","DOI":"10.1109\/GCCE50665.2020.9291748"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"207","DOI":"10.25101\/13.10","article-title":"A Case Study on Spotify: Exploring Perceptions of the Music Streaming Service","volume":"13","author":"Swanson","year":"2013","journal-title":"MEIEA J."},{"key":"ref_5","unstructured":"Korver, J. (2019). Facing the music: The Current State of Streaming Services in The Music Industry. [Ph.D. Thesis, Florida Southern College]."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Paul, D., and Kundu, S. (2020). A Survey of Music Recommendation Systems with a Proposed Music Recommendation System. Emerging Technology in Modelling and Graphics, Springer.","DOI":"10.1007\/978-981-13-7403-6_26"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Pichl, M., Zangerle, E., and Specht, G. (2016, January 11\u201313). Understanding playlist creation on music streaming platforms. Proceedings of the IEEE International Symposium on Multimedia, San Jose, CA, USA.","DOI":"10.1109\/ISM.2016.0107"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1080\/17530350.2019.1708780","article-title":"The editorial playlist as container technology: On Spotify and the logistical role of digital music packages","volume":"13","author":"Eriksson","year":"2020","journal-title":"J. Cult. Econ."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"765","DOI":"10.25300\/MISQ\/2019\/14750","article-title":"The Right Music at the Right Time: Adaptive Personalized Playlists Based on Sequence Modeling","volume":"43","author":"Liebman","year":"2019","journal-title":"MIS Q."},{"key":"ref_10","unstructured":"Schedl, M., Zamani, H., Chen, C.W., Deldjoo, Y., and Elahi, M. (2017, January 23\u201327). Recsys challenge 2018: Automatic playlist continuation. Proceedings of the Late-Breaking\/Demos 18th International Society for Music Information Retrieval Conference, Suzhou, China."},{"key":"ref_11","unstructured":"(2022, May 02). Music Biz Consumer Insights. Available online: https:\/\/musicbiz.org\/news\/playlists-overtake-albums-listenership-says-loop-study\/."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1007\/s11042-006-0003-9","article-title":"Automatic playlist generation based on tracking user\u2019s listening habits","volume":"29","author":"Andric","year":"2006","journal-title":"Multimed. Tools Appl."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"14375","DOI":"10.1007\/s11042-016-3836-x","article-title":"From manual to assisted playlist creation: A survey","volume":"76","author":"Dias","year":"2017","journal-title":"Multimed. Tools Appl."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Volkovs, M., Rai, H., Cheng, Z., Wu, G., Lu, Y., and Sanner, S. (2018). Two-stage model for automatic playlist continuation at scale. ACM Recommender Systems Challenge, ACM.","DOI":"10.1145\/3267471.3267480"},{"key":"ref_15","first-page":"1819","article-title":"A hybrid recommender system for improving automatic playlist continuation","volume":"33","author":"Gatzioura","year":"2019","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_16","unstructured":"Oliver, N., and Kreger-Stickles, L. (2006, January 8\u201312). PAPA: Physiology and Purpose-Aware Automatic Playlist Generation. Proceedings of the 7th International Society for Music Information Retrieval Conference, Victoria, BC, Canada."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Sakurai, K., Togo, R., Ogawa, T., and Haseyama, M. (2021, January 9\u201311). Music Playlist Generation Based on Graph Exploration Using Reinforcement Learning. Proceedings of the IEEE 3rd Global Conference on Life Sciences and Technologies, Nara, Japan.","DOI":"10.1109\/LifeTech52111.2021.9391870"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Vall, A. (2015, January 16\u201320). Listener-inspired automated music playlist generation. Proceedings of the 9th ACM Conference on Recommender Systems, Vienna, Austria.","DOI":"10.1145\/2792838.2796548"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Kaya, M., and Bridge, D. (2018). Automatic playlist continuation using subprofile-aware diversification. ACM Recommender Systems Challenge, ACM.","DOI":"10.1145\/3267471.3267472"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Nabizadeh, A.H., Jorge, A.M., Tang, S., and Yu, Y. (2016, January 20\u201322). Predicting user preference based on matrix factorization by exploiting music attributes. Proceedings of the 9th International Conference on Computer Science & Software Engineering, Porto, Portugal.","DOI":"10.1145\/2948992.2949010"},{"key":"ref_21","unstructured":"Choi, K., Fazekas, G., and Sandler, M. (2016). Towards playlist generation algorithms using rnns trained on within-track transitions. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Irene, R.T., Borrelli, C., Zanoni, M., Buccoli, M., and Sarti, A. (2019, January 2\u20136). Automatic playlist generation using convolutional neural networks and recurrent neural networks. Proceedings of the 27th European Signal Processing Conference, A Coru\u00f1a, Spain.","DOI":"10.23919\/EUSIPCO.2019.8903002"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Pauws, S., and Eggen, B. (2002, January 13\u201317). PATS: Realization and user evaluation of an automatic playlist generator. Proceedings of the 2nd International Society for Music Information Retrieval Conference, Paris, France.","DOI":"10.1076\/jnmr.32.2.179.16739"},{"key":"ref_24","unstructured":"Lee, J.H., Bare, B., and Meek, G. (2010, January 9\u201313). How Similar Is Too Similar?: Exploring Users\u2019 Perceptions of Similarity in Playlist Evaluation. Proceedings of the 11th International Society for Music Information Retrieval Conference, Utrecht, The Netherlands."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Wang, P., Fan, Y., Xia, L., Zhao, W.X., Niu, S., and Huang, J. (2020, January 25\u201330). KERL: A knowledge-guided reinforcement learning model for sequential recommendation. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Xi\u2019an, China.","DOI":"10.1145\/3397271.3401134"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"106706","DOI":"10.1016\/j.knosys.2020.106706","article-title":"A deep reinforcement learning based long-term recommender system","volume":"213","author":"Huang","year":"2021","journal-title":"Knowl.-Based Syst."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1613\/jair.301","article-title":"Reinforcement learning: A survey","volume":"4","author":"Kaelbling","year":"1996","journal-title":"J. Artif. Intell. Res."},{"key":"ref_28","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_29","unstructured":"Liebman, E., Saar-Tsechansky, M., and Stone, P. (2014). Dj-mc: A reinforcement-learning agent for music playlist recommendation. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Hu, B., Shi, C., and Liu, J. (2017, January 25\u201328). Playlist recommendation based on reinforcement learning. Proceedings of the International Conference on Intelligence Science, Shanghai, China.","DOI":"10.1007\/978-3-319-68121-4_18"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"He, R., and McAuley, J. (2016, January 12\u201315). Fusing similarity models with markov chains for sparse sequential recommendation. Proceedings of the IEEE 16th International Conference on Data Mining, Barcelona, Spain.","DOI":"10.1109\/ICDM.2016.0030"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Ge, Y., Zhao, S., Zhou, H., Pei, C., Sun, F., Ou, W., and Zhang, Y. (2020, January 25\u201330). Understanding echo chambers in e-commerce recommender systems. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Xi\u2019an, China.","DOI":"10.1145\/3397271.3401431"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Nguyen, T.T., Hui, P.M., Harper, F.M., Terveen, L., and Konstan, J.A. (2014, January 7\u201311). Exploring the filter bubble: The effect of using recommender systems on content diversity. Proceedings of the 23rd international Conference on World Wide Web, Seoul, Korea.","DOI":"10.1145\/2566486.2568012"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1109\/JPROC.2015.2483592","article-title":"A review of relational machine learning for knowledge graphs","volume":"104","author":"Nickel","year":"2015","journal-title":"IEEE"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Wang, Z., Zhang, J., Feng, J., and Chen, Z. (2014, January 27\u201331). Knowledge graph embedding by translating on hyperplanes. Proceedings of the AAAI Conference on Artificial Intelligence, Qu\u00e9bec City, QC, Canada.","DOI":"10.1609\/aaai.v28i1.8870"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Xian, Y., Fu, Z., Muthukrishnan, S., De Melo, G., and Zhang, Y. (2019, January 21\u201325). Reinforcement knowledge graph reasoning for explainable recommendation. Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Paris, France.","DOI":"10.1145\/3331184.3331203"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2652481","article-title":"Automated generation of music playlists: Survey and experiments","volume":"47","author":"Bonnin","year":"2014","journal-title":"ACM Comput. Surv."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Monti, D., Palumbo, E., Rizzo, G., Lisena, P., Troncy, R., Fell, M., Cabrio, E., and Morisio, M. (2018). An ensemble approach of recurrent neural networks using pre-trained embeddings for playlist completion. ACM Recommender Systems Challenge, ACM.","DOI":"10.1145\/3267471.3267484"},{"key":"ref_39","unstructured":"Vall, A., Quadrana, M., Schedl, M., and Widmer, G. (2018). The importance of song context and song order in automated music playlist generation. arXiv."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Kim, J., Won, M., Liem, C.C., and Hanjalic, A. (2018). Towards seed-free music playlist generation: Enhancing collaborative filtering with playlist title information. ACM Recommender Systems Challenge, ACM.","DOI":"10.1145\/3267471.3267485"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Yang, H., Zhao, Y., Xia, J., Yao, B., Zhang, M., and Zheng, K. (2019, January 22\u201325). Music playlist recommendation with long short-term memory. Proceedings of the International Conference on Database Systems for Advanced Applications, Chiang Mai, Thailand.","DOI":"10.1007\/978-3-030-18579-4_25"},{"key":"ref_42","unstructured":"Shih, S.Y., and Chi, H.Y. (2018). Automatic, personalized, and flexible playlist generation using reinforcement learning. arXiv."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Sun, Z., Yang, J., Zhang, J., Bozzon, A., Huang, L.K., and Xu, C. (2018, January 2). Recurrent knowledge graph embedding for effective recommendation. Proceedings of the 12th ACM Conference on Recommender Systems, Vancouver, BC, Canada.","DOI":"10.1145\/3240323.3240361"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"494","DOI":"10.1109\/TNNLS.2021.3070843","article-title":"A survey on knowledge graphs: Representation, acquisition, and applications","volume":"33","author":"Ji","year":"2021","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Wang, H., Zhang, F., Wang, J., Zhao, M., Li, W., Xie, X., and Guo, M. (2018, January 22\u201326). Ripplenet: Propagating user preferences on the knowledge graph for recommender systems. Proceedings of the 27th ACM International Conference on Information and Knowledge Management, Turin, Italy.","DOI":"10.1145\/3269206.3271739"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1109\/TKDE.2020.2981333","article-title":"Deep learning on graphs: A survey","volume":"34","author":"Zhang","year":"2020","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_47","unstructured":"Guo, Q., Zhuang, F., Qin, C., Zhu, H., Xie, X., Xiong, H., and He, Q. (2020). A survey on knowledge graph-based recommender systems. IEEE Trans. Knowl. Data Eng."},{"key":"ref_48","unstructured":"Song, W., Duan, Z., Yang, Z., Zhu, H., Zhang, M., and Tang, J. (2019). Explainable knowledge graph-based recommendation via deep reinforcement learning. arXiv."},{"key":"ref_49","first-page":"8","article-title":"Deep Reinforcement Learning-based Music Recommendation with Knowledge Graph Using Acoustic Features","volume":"10","author":"Sakurai","year":"2022","journal-title":"ITE Trans. Media Technol. Appl."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Zhou, S., Dai, X., Chen, H., Zhang, W., Ren, K., Tang, R., He, X., and Yu, Y. (2020, January 25\u201330). Interactive recommender system via knowledge graph-enhanced reinforcement learning. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Xi\u2019an, China.","DOI":"10.1145\/3397271.3401174"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Wang, X., Xu, Y., He, X., Cao, Y., Wang, M., and Chua, T.S. (2020, January 20\u201324). Reinforced negative sampling over knowledge graph for recommendation. Proceedings of the Web Conference, Taipei, Taiwan.","DOI":"10.1145\/3366423.3380098"},{"key":"ref_52","first-page":"2787","article-title":"Translating embeddings for modeling multi-relational data","volume":"26","author":"Bordes","year":"2013","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Cho, K., Van Merri\u00ebnboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014). Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv.","DOI":"10.3115\/v1\/D14-1179"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Ben-Elazar, S., Lavee, G., Koenigstein, N., Barkan, O., Berezin, H., Paquet, U., and Zaccai, T. (2017, January 6\u201310). Groove radio: A bayesian hierarchical model for personalized playlist generation. Proceedings of the ACM International Conference on Web Search and Data Mining, Cambridge, UK.","DOI":"10.1145\/3018661.3018718"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"290","DOI":"10.1111\/j.2517-6161.1976.tb01597.x","article-title":"The empirical distribution function with arbitrarily grouped, censored and truncated data","volume":"38","author":"Turnbull","year":"1976","journal-title":"J. R. Stat. Soc. Ser. B (Methodol.)"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Chen, C.W., Lamere, P., Schedl, M., and Zamani, H. (2018, January 2). Recsys challenge 2018: Automatic music playlist continuation. Proceedings of the 12th ACM Conference on Recommender Systems, Vancouver, BC, Canada.","DOI":"10.1145\/3240323.3240342"},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Ferraro, A., Kim, Y., Lee, S., Kim, B., Jo, N., Lim, S., Lim, S., Jang, J., Kim, S., and Serra, X. (2021, January 6\u201311). Melon Playlist Dataset: A public dataset for audio-based playlist generation and music tagging. Proceedings of the 46th IEEE International Conference on Acoustics, Speech and Signal Processing, Toronto, ON, Canada.","DOI":"10.1109\/ICASSP39728.2021.9413552"},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Jannach, D., Lerche, L., and Kamehkhosh, I. (2015, January 16\u201320). Beyond \u201chitting the hits\u201d Generating coherent music playlist continuations with the right tracks. Proceedings of the 9th ACM Conference on Recommender Systems, Vienna, Austria.","DOI":"10.1145\/2792838.2800182"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/10\/3722\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:10:22Z","timestamp":1760137822000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/10\/3722"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,5,13]]},"references-count":58,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2022,5]]}},"alternative-id":["s22103722"],"URL":"https:\/\/doi.org\/10.3390\/s22103722","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,5,13]]}}}