{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:12:43Z","timestamp":1750219963211,"version":"3.41.0"},"reference-count":59,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2024,7,31]],"date-time":"2024-07-31T00:00:00Z","timestamp":1722384000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"CNPq and FAPEMIG"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Recomm. Syst."],"published-print":{"date-parts":[[2024,12,31]]},"abstract":"<jats:p>Music streaming services heavily rely upon recommender systems to acquire, engage, and retain users. One notable component of these services are playlists, which can be dynamically generated in a sequential manner based on the user\u2019s feedback during a listening session. Online learning to rank approaches have recently been shown effective at leveraging such feedback to learn users\u2019 preferences in the space of song features. Nevertheless, these approaches can suffer from slow convergence as a result of their random exploration component and their session-agnostic exploitation component. To overcome these limitations, we propose a novel online learning to rank approach which efficiently explores the space of candidate recommendation models by restricting itself to the orthogonal complement of the subspace of previous underperforming exploration directions. Moreover, we propose a session-aware exploitation component which leverages the momentum of the current best model during updates. Our thorough evaluation using simulated listening sessions from two large Last.fm datasets demonstrates substantial improvements over state-of-the-art approaches in terms of early-stage performance, which results in an improved user experience during online learning. In addition, we demonstrate that long-term convergence can be further enhanced by adaptively relaxing exploration constraints along the way.<\/jats:p>","DOI":"10.1145\/3625827","type":"journal-article","created":{"date-parts":[[2023,9,27]],"date-time":"2023-09-27T08:35:56Z","timestamp":1695803756000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Efficient Exploration and Exploitation for Sequential Music Recommendation"],"prefix":"10.1145","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-2857-2913","authenticated-orcid":false,"given":"Bruno Laporais","family":"Pereira","sequence":"first","affiliation":[{"name":"Department of Computer Science, Universidade Federal de Minas Gerais, Belo Horizonte, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-4595-7680","authenticated-orcid":false,"given":"Pedro Dalla Vecchia","family":"Chaves","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Universidade Federal de Minas Gerais, Belo Horizonte, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2921-8444","authenticated-orcid":false,"given":"Rodrygo L. T.","family":"Santos","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Universidade Federal de Minas Gerais, Belo Horizonte, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,7,31]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3306618.3314309"},{"key":"e_1_3_2_3_2","doi-asserted-by":"crossref","first-page":"286","DOI":"10.1007\/11805816_22","volume-title":"Proceedings of the European Conference on Case-Based Reasoning","author":"Baccigalupo Claudio","year":"2006","unstructured":"Claudio Baccigalupo and Enric Plaza. 2006. Case-based sequential ordering of songs for playlist recommendation. In Proceedings of the European Conference on Case-Based Reasoning, Thomas R. Roth-Berghofer, Mehmet H. G\u00f6ker, and H. Altay G\u00fcvenir (Eds.). Springer, Berlin, 286\u2013300."},{"key":"e_1_3_2_4_2","first-page":"115","volume-title":"Proceedings of the 30th International Conference on Machine Learning","volume":"28","author":"Bergstra James","year":"2013","unstructured":"James Bergstra, Daniel Yamins, and David Cox. 2013. Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures. In Proceedings of the 30th International Conference on Machine Learning. Sanjoy Dasgupta and David McAllester (Eds.), PMLR, Vol. 28, Atlanta, GA, 115\u2013123."},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/2652481"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.fss.2009.05.013"},{"key":"e_1_3_2_7_2","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-13287-2","volume-title":"Music Recommendation and Discovery: The Long Tail, Long Fail, and Long Play in the Digital Music Space (1st ed.)","author":"Celma Oscar","year":"2010","unstructured":"Oscar Celma. 2010. Music Recommendation and Discovery: The Long Tail, Long Fail, and Long Play in the Digital Music Space (1st ed.). Springer, Berlin."},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/3485447.3512116"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/2339530.2339643"},{"key":"e_1_3_2_10_2","first-page":"1","article-title":"Fairness in recommender systems: Research landscape and future directions","author":"Deldjoo Yashar","year":"2023","unstructured":"Yashar Deldjoo, Dietmar Jannach, Alejandro Bellogin, Alessandro Difonzo, and Dario Zanzonelli. 2023. Fairness in recommender systems: Research landscape and future directions. User Modeling and User-Adapted Interaction (2023), 1\u201350. https:\/\/link.springer.com\/article\/10.1007\/s11257-023-09364-z#article-info","journal-title":"User Modeling and User-Adapted Interaction"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-016-3836-x"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-4380-9_41"},{"key":"e_1_3_2_13_2","first-page":"385","volume-title":"Proceedings of the 16th Annual ACM-SIAM Symposium on Discrete Algorithms","author":"Flaxman Abraham D.","year":"2005","unstructured":"Abraham D. Flaxman, Adam Tauman Kalai, and H. Brendan McMahan. 2005. Online convex optimization in the bandit setting: Gradient descent without a gradient. In Proceedings of the 16th Annual ACM-SIAM Symposium on Discrete Algorithms. Society for Industrial and Applied Mathematics, 385\u2013394."},{"key":"e_1_3_2_14_2","first-page":"I\u2013229","volume-title":"Proceedings of the 2007 IEEE International Conference on Acoustics, Speech and Signal Processing","volume":"1","author":"Gartner Don","year":"2007","unstructured":"Don Gartner, Florian Kraft, and Thomas Schaaf. 2007. An adaptive distance measure for similarity based playlist generation. In Proceedings of the 2007 IEEE International Conference on Acoustics, Speech and Signal Processing. Vol. 1, IEEE, I\u2013229."},{"key":"e_1_3_2_15_2","first-page":"1253","volume-title":"Proceedings of the 34th International Conference on Machine Learning.","volume":"70","author":"Gentile Claudio","year":"2017","unstructured":"Claudio Gentile, Shuai Li, Purushottam Kar, Alexandros Karatzoglou, Giovanni Zappella, and Evans Etrue. 2017. On context-dependent clustering of bandits. In Proceedings of the 34th International Conference on Machine Learning. Vol. 70, PMLR, 1253\u20131262."},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/2911451.2914798"},{"issue":"5","key":"e_1_3_2_17_2","doi-asserted-by":"crossref","first-page":"625","DOI":"10.1080\/03007766.2015.1021174","article-title":"The playlist experience: Personal playlists in music streaming services","volume":"38","author":"Hagen Anja Nylund","year":"2015","unstructured":"Anja Nylund Hagen. 2015. The playlist experience: Personal playlists in music streaming services. Popular Music and Society 38, 5 (2015), 625\u2013645.","journal-title":"Popular Music and Society"},{"key":"e_1_3_2_18_2","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1145\/2365952.2365979","volume-title":"Proceedings of the 6th ACM Conference on Recommender Systems","author":"Hariri Negar","year":"2012","unstructured":"Negar Hariri, Bamshad Mobasher, and Robin Burke. 2012. Context-aware music recommendation based on latenttopic sequential patterns. In Proceedings of the 6th ACM Conference on Recommender Systems. ACM, New York, NY, 131\u2013138."},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.5555\/2832747.2832852"},{"key":"e_1_3_2_20_2","first-page":"123","volume-title":"Proceedings of the International Conference on IEEE Web Delivering of Music","author":"Hauver David B.","year":"2001","unstructured":"David B. Hauver and James C. French. 2001. Flycasting: Using collaborative filtering to generate a playlist for online radio. In Proceedings of the International Conference on IEEE Web Delivering of Music. IEEE, 123\u2013130."},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/2568388.2568413"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/2433396.2433419"},{"volume-title":"Intelligence Science I","key":"e_1_3_2_23_2","unstructured":"Binbin Hu, Chuan Shi, and Jian Liu. 2017. Playlist recommendation based on reinforcement learning. In Intelligence Science I, Springer International Publishing, Cham, 172-182."},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2008.22"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/1869446.1869453"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-019-09237-4"},{"key":"e_1_3_2_27_2","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1007\/978-3-319-16354-3_34","volume-title":"Advances in Information Retrieval","author":"King James","year":"2015","unstructured":"James King and Vaiva Imbrasait\u0117. 2015. Generating music playlists with hierarchical clustering and Q-learning. In Advances in Information Retrieval, Allan Hanbury, Gabriella Kazai, Andreas Rauber, and Norbert Fuhr (Eds.). Springer International Publishing, Cham, 315\u2013326."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772758"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/1935826.1935878"},{"key":"e_1_3_2_30_2","first-page":"591","volume-title":"Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems","author":"Liebman Elad","year":"2015","unstructured":"Elad Liebman, Maytal Saar-Tsechansky, and Peter Stone. 2015. DJ-MC: A Reinforcement-learning agent for music playlist recommendation. In Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems. ACM, 591\u2013599."},{"key":"e_1_3_2_31_2","first-page":"295","volume-title":"Proceedings of the 3rd International Conference on Music Information Retrieval","volume":"2","author":"Logan Beth","year":"2002","unstructured":"Beth Logan. 2002. Content-based playlist generation: Exploratory experiments. In Proceedings of the 3rd International Conference on Music Information Retrieval. Vol. 2, 295\u2013296."},{"key":"e_1_3_2_32_2","first-page":"349","volume-title":"Proceedings of the 13th International Society for Music Information Retrieval Conference","author":"Moore Joshua L.","year":"2012","unstructured":"Joshua L. Moore, Shuo Chen, Thorsten Joachims, and Douglas Turnbull. 2012. Learning to embed songs and tags for playlist prediction. In Proceedings of the 13th International Society for Music Information Retrieval Conference. 349\u2013354."},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3511808.3557656"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3132847.3132896"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3269206.3271686"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/1571941.1572030"},{"key":"e_1_3_2_37_2","first-page":"634","volume-title":"Proceedings of the 6th International Conference on Music Information Retrieval","author":"Pampalk Elias","year":"2005","unstructured":"Elias Pampalk, Tim Pohle, and Gerhard Widmer. 2005. Dynamic playlist generation based on skipping behavior. In Proceedings of the 6th International Conference on Music Information Retrieval. 634\u2013637."},{"issue":"2","key":"e_1_3_2_38_2","doi-asserted-by":"crossref","first-page":"184","DOI":"10.3384\/cu.2000.1525.1792184","article-title":"More of the Same\u2014On Spotify Radio","volume":"9","author":"Pelle Snickars","year":"2017","unstructured":"Snickars Pelle. 2017. More of the Same\u2014On Spotify Radio. Culture Unbound. Journal of Current Cultural Research 9, 2 (2017), 184\u2013211.","journal-title":"Culture Unbound. Journal of Current Cultural Research"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/3298689.3347019"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.4018\/IJMDEM.2017100103"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1016\/0041-5553(64)90137-5"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3190616"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/1458082.1458092"},{"key":"e_1_3_2_44_2","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1145\/3498366.3505791","volume-title":"Proceedings of the ACM SIGIR Conference on Human Information Interaction and Retrieval","author":"Schedl Markus","year":"2022","unstructured":"Markus Schedl, Stefan Brandl, Oleg Lesota, Emilia Parada-Cabaleiro, David Penz, and Navid Rekabsaz. 2022. LFM-2b: A dataset of enriched music listening events for recommender systems research and fairness analysis. In Proceedings of the ACM SIGIR Conference on Human Information Interaction and Retrieval. ACM, New York, NY, 337\u2013341."},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/2835776.2835804"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/2661829.2661952"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/2792838.2796548"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3167132.3167280"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/2766462.2767716"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331264"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3210045"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3465401"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/3159652.3159732"},{"issue":"1","key":"e_1_3_2_54_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2623372","article-title":"Exploration in interactive personalized music recommendation: A reinforcement learning approach","volume":"11","author":"Wang Xinxi","year":"2014","unstructured":"Xinxi Wang, Yi Wang, David Hsu, and Ye Wang. 2014. Exploration in interactive personalized music recommendation: A reinforcement learning approach. ACM Transactions on Multimedia Computing, Communications, and Applications 11, 1 (2014), 1\u201322.","journal-title":"ACM Transactions on Multimedia Computing, Communications, and Applications"},{"key":"e_1_3_2_55_2","first-page":"445","volume-title":"Proceedings of the 15th International Society for Music Information Retrieval Conference","author":"Xing Zhe","year":"2014","unstructured":"Zhe Xing, Xinxi Wang, and Ye Wang. 2014. Enhancing collaborative filtering music recommendation by balancing exploration and exploitation. In Proceedings of the 15th International Society for Music Information Retrieval Conference. 445\u2013450."},{"key":"e_1_3_2_56_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Xiong Lee","year":"2021","unstructured":"Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N. Bennett, Junaid Ahmed, and Arnold Overwijk. 2021. Approximate nearest neighbor negative contrastive learning for dense text retrieval. In Proceedings of the International Conference on Learning Representations."},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/1553374.1553527"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462880"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/2983323.2983774"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462937"}],"container-title":["ACM Transactions on Recommender Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3625827","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3625827","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:49:12Z","timestamp":1750182552000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3625827"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,31]]},"references-count":59,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,12,31]]}},"alternative-id":["10.1145\/3625827"],"URL":"https:\/\/doi.org\/10.1145\/3625827","relation":{},"ISSN":["2770-6699"],"issn-type":[{"type":"electronic","value":"2770-6699"}],"subject":[],"published":{"date-parts":[[2024,7,31]]},"assertion":[{"value":"2022-11-06","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-09-11","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-07-31","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}