{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T07:42:38Z","timestamp":1740123758154,"version":"3.37.3"},"reference-count":52,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2021,11,6]],"date-time":"2021-11-06T00:00:00Z","timestamp":1636156800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,11,6]],"date-time":"2021-11-06T00:00:00Z","timestamp":1636156800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Hungarian Ministry of Innovation and Technology NRDI Office within the framework of the Hungarian Artificial Intelligence National Laboratory Program"},{"name":"ELKH Institute for Computer Science and Control"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["User Model User-Adap Inter"],"published-print":{"date-parts":[[2022,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>As a task of high importance for recommender systems, we consider the problem of learning the convex combination of ranking algorithms by online machine learning. First, we propose a stochastic optimization algorithm that uses finite differences. Our new algorithm achieves close to optimal empirical performance for two base rankers, while scaling well with an increased number of models. In our experiments with five real-world recommendation data sets, we show that the combination offers significant improvement over previously known stochastic optimization techniques. The proposed algorithm is the first effective stochastic optimization method for combining ranked recommendation lists by online machine learning. Secondly, we propose an exponentially weighted algorithm based on a grid over the space of combination weights. We show that the algorithm has near-optimal worst-case performance bound. The bound provides the first theoretical guarantee for non-convex bandits using limited number of evaluations under very general conditions.<\/jats:p>","DOI":"10.1007\/s11257-021-09306-7","type":"journal-article","created":{"date-parts":[[2021,11,6]],"date-time":"2021-11-06T16:02:31Z","timestamp":1636214551000},"page":"649-683","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Online convex combination of ranking models"],"prefix":"10.1007","volume":"32","author":[{"given":"Erzs\u00e9bet","family":"Frig\u00f3","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8700-5060","authenticated-orcid":false,"given":"Levente","family":"Kocsis","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,11,6]]},"reference":[{"unstructured":"Abernethy, J., Canini, K., Langford, J., Simma, A.: Online Collaborative Filtering. University of California at Berkeley, Technical Report (2007)","key":"9306_CR1"},{"unstructured":"Agarwal, A., Dekel, O., Xiao, L.: Optimal algorithms for online convex optimization with multi-point bandit feedback. In: COLT, pp. 28\u201340 (2010)","key":"9306_CR2"},{"doi-asserted-by":"crossref","unstructured":"Al-Ghossein, M., Murena, P.A., Abdessalem, T., Barr\u00e9, A., Cornu\u00e9jols, A.: Adaptive collaborative topic modeling for online recommendation. In: Proceedings of the 12th ACM Conference on Recommender Systems, pp. 338\u2013346. ACM (2018)","key":"9306_CR3","DOI":"10.1145\/3240323.3240363"},{"doi-asserted-by":"crossref","unstructured":"Amatriain, X., Agarwal, D.: Tutorial: lessons learned from building real-life recommender systems. In: Proceedings of the 10th ACM Conference on Recommender Systems, p. 433 (2016)","key":"9306_CR4","DOI":"10.1145\/2959100.2959194"},{"doi-asserted-by":"crossref","unstructured":"Au, C.K., Leung, H.F.: An empirical comparison of CMA-ES in dynamic environments. In: International Conference on Parallel Problem Solving from Nature, pp. 529\u2013538. Springer (2012)","key":"9306_CR5","DOI":"10.1007\/978-3-642-32937-1_53"},{"issue":"1","key":"9306_CR6","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1137\/S0097539701398375","volume":"32","author":"P Auer","year":"2002","unstructured":"Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM J. Comput. 32(1), 48\u201377 (2002)","journal-title":"SIAM J. Comput."},{"unstructured":"Balcan, M.F., Dick, T., Sharma, D.: Online optimization of piecewise Lipschitz functions in changing environments (2019). arXiv:1907.09137","key":"9306_CR7"},{"doi-asserted-by":"crossref","unstructured":"Bennett, J., Lanning, S., et\u00a0al.: The Netflix prize. In: Proceedings of KDD Cup and Workshop, vol. 2007, p.\u00a035. New York, NY, USA (2007)","key":"9306_CR8","DOI":"10.1145\/1345448.1345459"},{"unstructured":"Bubeck, S., Munos, R., Stoltz, G., Szepesv\u00e1ri, C.: X-armed bandits. J. Mach. Learn. Res. 12(5) (2011)","key":"9306_CR9"},{"doi-asserted-by":"crossref","unstructured":"Burke, R.: Evaluating the dynamic properties of recommendation algorithms. In: Proceedings of the fourth ACM Conference on Recommender Systems, pp. 225\u2013228. ACM (2010)","key":"9306_CR10","DOI":"10.1145\/1864708.1864753"},{"unstructured":"Busa-Fekete, R., K\u00e9gl, B., \u00c9ltet\u0151, T., Szarvas, G.: Ranking by calibrated adaboost. In: Proceedings of the Learning to Rank Challenge, pp. 37\u201348 (2011)","key":"9306_CR11"},{"key":"9306_CR12","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511546921","volume-title":"Prediction, Learning, and Games","author":"N Cesa-Bianchi","year":"2006","unstructured":"Cesa-Bianchi, N., Lugosi, G.: Prediction, Learning, and Games. Cambridge University Press, Cambridge (2006)"},{"unstructured":"Cohen-Addad, V., Kanade, V.: Online optimization of smoothed piecewise constant functions. In: Artificial Intelligence and Statistics, pp. 412\u2013420. PMLR (2017)","key":"9306_CR13"},{"key":"9306_CR14","doi-asserted-by":"publisher","DOI":"10.1137\/1.9780898718768","volume-title":"Introduction to Derivative-Free Optimization","author":"AR Conn","year":"2009","unstructured":"Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization. SIAM, Philadelphia (2009)"},{"doi-asserted-by":"crossref","unstructured":"Craswell, N., Zoeter, O., Taylor, M., Ramsey, B.: An experimental comparison of click position-bias models. In: Proceedings of the 2008 International Conference on Web Search and Data Mining, pp. 87\u201394 (2008)","key":"9306_CR15","DOI":"10.1145\/1341531.1341545"},{"doi-asserted-by":"crossref","unstructured":"Gama, J., Sebasti\u00e3o, R., Rodrigues, P.P.: Issues in evaluation of stream learning algorithms. In: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 329\u2013338. ACM (2009)","key":"9306_CR16","DOI":"10.1145\/1557019.1557060"},{"issue":"2","key":"9306_CR17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3054925","volume":"50","author":"HM Gomes","year":"2017","unstructured":"Gomes, H.M., Barddal, J.P., Enembreck, F., Bifet, A.: A survey on ensemble learning for data stream classification. ACM Comput. Surv. (CSUR) 50(2), 1\u201336 (2017)","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"9306_CR18","first-page":"667","volume":"28","author":"JB Grill","year":"2015","unstructured":"Grill, J.B., Valko, M., Munos, R.: Black-box optimization of noisy functions with unknown smoothness. Adv. Neural Inf. Process. Syst. 28, 667\u2013675 (2015)","journal-title":"Adv. Neural Inf. Process. Syst."},{"doi-asserted-by":"crossref","unstructured":"Hansen, N., Auger, A., Ros, R., Finck, S., Po\u0161\u00edk, P.: Comparing results of 31 algorithms from the black-box optimization benchmarking bbob-2009. In: Proceedings of the 12th Annual Conference Companion on Genetic and Evolutionary Computation, pp. 1689\u20131696 (2010)","key":"9306_CR19","DOI":"10.1145\/1830761.1830790"},{"key":"9306_CR20","first-page":"784","volume":"27","author":"E Hazan","year":"2014","unstructured":"Hazan, E., Levy, K.: Bandit convex optimization: towards tight bounds. Adv. Neural Inf. Process. Syst. 27, 784\u2013792 (2014)","journal-title":"Adv. Neural Inf. Process. Syst."},{"unstructured":"Hazan, E., Li, Y.: An optimal algorithm for bandit convex optimization (2016). arXiv:1603.04350","key":"9306_CR21"},{"doi-asserted-by":"crossref","unstructured":"Hu, Y., Koren, Y., Volinsky, C.: Collaborative filtering for implicit feedback datasets. In: 2008 Eighth IEEE International Conference on Data Mining, pp. 263\u2013272 (2008)","key":"9306_CR22","DOI":"10.1109\/ICDM.2008.22"},{"unstructured":"Igel, C., H\u00fcsken, M.: Improving the Rprop learning algorithm. In: Bothe, H., Rojas,, R. (eds.) Proceedings of the Second International ICSC Symposium on Neural Computation (NC 2000), pp. 115\u2013121. ICSC Academic Press (2000)","key":"9306_CR23"},{"doi-asserted-by":"crossref","unstructured":"Igel, C., Suttorp, T., Hansen, N.: A computational efficient covariance matrix update and a (1+1)-CMA for evolution strategies. In: Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation, pp. 453\u2013460 (2006)","key":"9306_CR24","DOI":"10.1145\/1143997.1144082"},{"doi-asserted-by":"crossref","unstructured":"J\u00e4rvelin, K., Kek\u00e4l\u00e4inen, J.: IR evaluation methods for retrieving highly relevant documents. In: Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 41\u201348. ACM (2000)","key":"9306_CR25","DOI":"10.1145\/345508.345545"},{"doi-asserted-by":"crossref","unstructured":"Jugovac, M., Jannach, D., Karimi, M.: Streamingrec: a framework for benchmarking stream-based news recommenders. In: Proceedings of the 12th ACM Conference on Recommender Systems, pp. 269\u2013273. ACM (2018)","key":"9306_CR26","DOI":"10.1145\/3240323.3240384"},{"unstructured":"Kleinberg, R.D.: Nearly tight bounds for the continuum-armed bandit problem. In: Advances in Neural Information Processing Systems, pp. 697\u2013704 (2005)","key":"9306_CR27"},{"issue":"3","key":"9306_CR28","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1007\/s10994-006-6888-8","volume":"63","author":"L Kocsis","year":"2006","unstructured":"Kocsis, L., Szepesv\u00e1ri, C.: Universal parameter optimisation in games based on SPSA. Mach. Learn. 63(3), 249\u2013286 (2006)","journal-title":"Mach. Learn."},{"doi-asserted-by":"crossref","unstructured":"Koren, Y.: Factorization meets the neighborhood: a multifaceted collaborative filtering model. In: Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 426\u2013434. ACM (2008)","key":"9306_CR29","DOI":"10.1145\/1401890.1401944"},{"issue":"8","key":"9306_CR30","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1109\/MC.2009.263","volume":"42","author":"Y Koren","year":"2009","unstructured":"Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42(8), 30\u201337 (2009)","journal-title":"Computer"},{"key":"9306_CR31","doi-asserted-by":"crossref","DOI":"10.1002\/9781118914564","volume-title":"Combining Pattern Classifiers: Methods and Algorithms","author":"LI Kuncheva","year":"2014","unstructured":"Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. Wiley, New York (2014)"},{"unstructured":"Kveton, B., Szepesvari, C., Wen, Z., Ashkan, A.: Cascading bandits: learning to rank in the cascade model. In: International Conference on Machine Learning, pp. 767\u2013776 (2015)","key":"9306_CR32"},{"doi-asserted-by":"crossref","unstructured":"Larson, J., Menickelly, M., Wild, S.M.: Derivative-free optimization methods (2019). arXiv:1904.11585","key":"9306_CR33","DOI":"10.1017\/S0962492919000060"},{"doi-asserted-by":"crossref","unstructured":"Lathia, N., Hailes, S., Capra, L.: Temporal collaborative filtering with adaptive neighbourhoods. In: Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 796\u2013797. ACM (2009)","key":"9306_CR34","DOI":"10.1145\/1571941.1572133"},{"doi-asserted-by":"crossref","unstructured":"Maillard, O.A., Munos, R.: Online learning in adversarial lipschitz environments. In: Machine Learning and Knowledge Discovery in Databases, pp. 305\u2013320 (2010)","key":"9306_CR35","DOI":"10.1007\/978-3-642-15883-4_20"},{"doi-asserted-by":"crossref","unstructured":"McAuley, J., Targett, C., Shi, Q., Den Hengel, Van, A.: Image-based recommendations on styles and substitutes. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 43\u201352. ACM (2015)","key":"9306_CR36","DOI":"10.1145\/2766462.2767755"},{"key":"9306_CR37","first-page":"3168","volume":"28","author":"G Neu","year":"2015","unstructured":"Neu, G.: Explore no more: improved high-probability regret bounds for non-stochastic bandits. Adv. Neural Inf. Process. Syst. 28, 3168\u20133176 (2015)","journal-title":"Adv. Neural Inf. Process. Syst."},{"issue":"1","key":"9306_CR38","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1007\/s13278-014-0244-y","volume":"5","author":"R P\u00e1lovics","year":"2015","unstructured":"P\u00e1lovics, R., Bencz\u00far, A.A.: Temporal influence over the Last.fm social network. Soc. Netw. Anal. Min. 5(1), 4 (2015)","journal-title":"Soc. Netw. Anal. Min."},{"doi-asserted-by":"crossref","unstructured":"P\u00e1lovics, R., Bencz\u00far, A.A., Kocsis, L., Kiss, T., Frig\u00f3, E.:ACM, : Exploiting temporal influence in online recommendation. In: Proceedings of the 8th ACM Conference on Recommender Systems, pp. 273\u2013280. ACM (2014)","key":"9306_CR39","DOI":"10.1145\/2645710.2645723"},{"key":"9306_CR40","doi-asserted-by":"publisher","first-page":"490","DOI":"10.1016\/j.pmcj.2016.06.001","volume":"38","author":"R P\u00e1lovics","year":"2017","unstructured":"P\u00e1lovics, R., Szalai, P., Pap, J., Frig\u00f3, E., Kocsis, L., Bencz\u00far, A.A.: Location-aware online learning for top-k recommendation. Pervasive Mob. Comput. 38, 490\u2013504 (2017)","journal-title":"Pervasive Mob. Comput."},{"doi-asserted-by":"crossref","unstructured":"Pan, R., Zhou, Y., Cao, B., Liu, N.N., Lukose, R., Scholz, M., Yang, Q.: One-class collaborative filtering. In: Eighth IEEE International Conference on Data Mining, 2008. ICDM\u201908, pp. 502\u2013511. IEEE (2008)","key":"9306_CR41","DOI":"10.1109\/ICDM.2008.16"},{"unstructured":"Pil\u00e1szy, I., Ser\u00e9ny, A., D\u00f3zsa, G., Hidasi, B., S\u00e1ri, A., Gub, J.: Neighbor methods vs. matrix factorizationcase studies of real-life recommendations. In: LSRS Workshop at ACM RecSys (2015)","key":"9306_CR42"},{"doi-asserted-by":"crossref","unstructured":"Radlinski, F., Kleinberg, R., Joachims, T.: Learning diverse rankings with multi-armed bandits. In: Proceedings of the 25th International Conference on Machine Learning, pp. 784\u2013791. ACM (2008)","key":"9306_CR43","DOI":"10.1145\/1390156.1390255"},{"doi-asserted-by":"crossref","unstructured":"Sarwar, B., Karypis, G., Konstan, J., Riedl, J.: Item-based collaborative filtering recommendation algorithms. In: Proceedings of the 10th International Conference on World Wide Web, pp. 285\u2013295. ACM (2001)","key":"9306_CR44","DOI":"10.1145\/371920.372071"},{"unstructured":"Seldin, Y., Bartlett, P., Crammer, K., Abbasi-Yadkori, Y.: Prediction with limited advice and multiarmed bandits with paid observations. In: ICML, pp. 280\u2013287 (2014)","key":"9306_CR45"},{"issue":"1","key":"9306_CR46","first-page":"1703","volume":"18","author":"O Shamir","year":"2017","unstructured":"Shamir, O.: An optimal algorithm for bandit and zero-order convex optimization with two-point feedback. J. Mach. Learn. Res. 18(1), 1703\u20131713 (2017)","journal-title":"J. Mach. Learn. Res."},{"key":"9306_CR47","doi-asserted-by":"publisher","first-page":"332","DOI":"10.1109\/9.119632","volume":"37","author":"JC Spall","year":"1992","unstructured":"Spall, J.C.: Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans. Autom. Control 37, 332\u2013341 (1992)","journal-title":"IEEE Trans. Autom. Control"},{"doi-asserted-by":"crossref","unstructured":"T\u00f6scher, A., Jahrer, M., Bell, R.M.: The BigChaos solution to the Netflix grand prize. Netflix prize documentation, pp. 1\u201352. (2009)","key":"9306_CR48","DOI":"10.1145\/1722149.1722153"},{"unstructured":"Vinagre, J., Jorge, A.M., Gama, J.: Evaluation of recommender systems in streaming environments. In: Workshop on \u2019Recommender Systems Evaluation: Dimensions and Design\u2019 (REDD 2014), held in conjunction with RecSys 2014 (2014)","key":"9306_CR49"},{"doi-asserted-by":"crossref","unstructured":"Voorhees, E.M., Tice, D.M.: The TREC-8 question answering track report. In: TREC, vol. 99, pp. 77\u201382 (1999)","key":"9306_CR50","DOI":"10.6028\/NIST.SP.500-246.qa-overview"},{"doi-asserted-by":"crossref","unstructured":"Yue, Y., Joachims, T.: Interactively optimizing information retrieval systems as a dueling bandits problem. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 1201\u20131208. ACM (2009)","key":"9306_CR51","DOI":"10.1145\/1553374.1553527"},{"unstructured":"Zoller, D., Doerfel, S., P\u00f6litz, C., Hotho, A.: Leveraging user-interactions for time-aware tag recommendations. In: RecTemp@ RecSys, pp. 9\u201315 (2017)","key":"9306_CR52"}],"container-title":["User Modeling and User-Adapted Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11257-021-09306-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11257-021-09306-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11257-021-09306-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,11]],"date-time":"2024-09-11T17:12:08Z","timestamp":1726074728000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11257-021-09306-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,6]]},"references-count":52,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,9]]}},"alternative-id":["9306"],"URL":"https:\/\/doi.org\/10.1007\/s11257-021-09306-7","relation":{},"ISSN":["0924-1868","1573-1391"],"issn-type":[{"type":"print","value":"0924-1868"},{"type":"electronic","value":"1573-1391"}],"subject":[],"published":{"date-parts":[[2021,11,6]]},"assertion":[{"value":"5 March 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 September 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 November 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}