{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T07:31:26Z","timestamp":1740123086722,"version":"3.37.3"},"reference-count":47,"publisher":"Springer Science and Business Media LLC","issue":"7","license":[{"start":{"date-parts":[[2021,11,9]],"date-time":"2021-11-09T00:00:00Z","timestamp":1636416000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,11,9]],"date-time":"2021-11-09T00:00:00Z","timestamp":1636416000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Mach Learn"],"published-print":{"date-parts":[[2022,7]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We outline an inherent flaw of tensor factorization models when latent factors are expressed as a function of side information and propose a novel method to mitigate this. We coin our methodology <jats:italic>kernel fried tensor<\/jats:italic> (KFT) and present it as a large-scale prediction and forecasting tool for high dimensional data. Our results show superior performance against <jats:italic>LightGBM<\/jats:italic> and <jats:italic>Field aware factorization machines<\/jats:italic> (FFM), two algorithms with proven track records, widely used in large-scale prediction. We also develop a variational inference framework for KFT which enables associating the predictions and forecasts with calibrated uncertainty estimates on several datasets.<\/jats:p>","DOI":"10.1007\/s10994-021-06067-7","type":"journal-article","created":{"date-parts":[[2021,11,9]],"date-time":"2021-11-09T23:02:27Z","timestamp":1636498947000},"page":"2663-2713","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Large scale tensor regression using kernels and variational inference"],"prefix":"10.1007","volume":"111","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8620-4618","authenticated-orcid":false,"given":"Robert","family":"Hu","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Geoff K.","family":"Nicholls","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dino","family":"Sejdinovic","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,11,9]]},"reference":[{"doi-asserted-by":"publisher","unstructured":"Agarwal, D., & Chen, B. C. (2009). Regression-based latent factor models (Vol. \u201909, pp. 19\u201328). Association for Computing Machinery. https:\/\/doi.org\/10.1145\/1557019.1557029","key":"6067_CR1","DOI":"10.1145\/1557019.1557029"},{"unstructured":"Batselier, K. (2018). The trouble with tensor ring decompositions. Preprint","key":"6067_CR2"},{"unstructured":"Bergstra, J., Yamins, D., Cox, D. D. (2013) Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures. Proc. of the 30th International Conference on Machine Learning (ICML 2013).","key":"6067_CR3"},{"key":"6067_CR4","doi-asserted-by":"publisher","first-page":"225","DOI":"10.1016\/j.knosys.2011.07.021","volume":"26","author":"J Bobadilla","year":"2012","unstructured":"Bobadilla, J., Ortega, F., Hernando, A., & Bernal, J. (2012). A collaborative filtering approach to mitigate the new user cold start problem. Knowledge-Based Systems, 26, 225\u2013238. https:\/\/doi.org\/10.1016\/j.knosys.2011.07.021","journal-title":"Knowledge-Based Systems"},{"key":"6067_CR5","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1016\/j.knosys.2013.03.012","volume":"46","author":"J Bobadilla","year":"2013","unstructured":"Bobadilla, J., Ortega, F., Hernando, A., & Guti\u00e9rrez, A. (2013). Recommender systems survey. Knowledge-Based Systems, 46, 109\u2013132. https:\/\/doi.org\/10.1016\/j.knosys.2013.03.012","journal-title":"Knowledge-Based Systems"},{"key":"6067_CR6","doi-asserted-by":"publisher","DOI":"10.1287\/opre.1090.0698","author":"F Caro","year":"2010","unstructured":"Caro, F., & Gallien, J. (2010). Inventory management of a fast-fashion retail network. Operations Research. https:\/\/doi.org\/10.1287\/opre.1090.0698","journal-title":"Operations Research"},{"doi-asserted-by":"crossref","unstructured":"Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society. Series B (Methodological) 39(1), 1\u201338. http:\/\/www.jstor.org\/stable\/2984875","key":"6067_CR7","DOI":"10.1111\/j.2517-6161.1977.tb01600.x"},{"unstructured":"Draper, N., & Smith, H. (1966). Applied regression analysis. Wiley series in probability and mathematical statistics. Wiley. http:\/\/gso.gbv.de\/DB=2.1\/CMD?ACT=SRCHA&SRT=YOP&IKT=1016&TRM=ppn+022791892&sourceid=fbw_bibsonomy","key":"6067_CR8"},{"doi-asserted-by":"crossref","unstructured":"Du, Chao, Chongxuan Li, Yin Zheng, Jun Zhu, and Bo Zhang. (2018). Collaborative Filtering With User-Item Co-Autoregressive Models. Proceedings of the AAAI Conference on Artificial Intelligence 32(1). https:\/\/ojs.aaai.org\/index.php\/AAAI\/article\/view\/11884.","key":"6067_CR9","DOI":"10.1609\/aaai.v32i1.11884"},{"unstructured":"G\u00f6nen, M., Khan, S., & Kaski, S. (2013). In S. Dasgupta & D. McAllester (Eds.), Kernelized Bayesian matrix factorization (Vol. 28, pp. 864\u2013872). PMLR. http:\/\/proceedings.mlr.press\/v28\/gonen13a.html","key":"6067_CR10"},{"unstructured":"Hawkins, C., & Zhang, Z. (2018). Variational Bayesian Inference for Robust Streaming Tensor Factorization and Completion. 2018 IEEE International Conference on Data Mining (ICDM), 1446\u20131451.","key":"6067_CR11"},{"unstructured":"He, L., Lu, C. T., Ma, G., Wang, S., Shen, L., Yu, P. S., & Ragin, A. B. (2017). In D. Precup & Y. W. Teh (Eds.) Kernelized support tensor machines (Vol. 70, pp. 1442\u20131451). PMLR, International Convention Centre. http:\/\/proceedings.mlr.press\/v70\/he17a.html","key":"6067_CR12"},{"unstructured":"Hoffman, M. D., Blei, D. M., Wang, C., & Paisley, J. (2013). Stochastic variational inference. Journal of Machine Learning Research, 14(1), 1303\u20131347. http:\/\/dl.acm.org\/citation.cfm?id=2502581.2502622","key":"6067_CR13"},{"doi-asserted-by":"publisher","unstructured":"Juan, Y., Zhuang, Y., Chin, W. S., & Lin, C. J. (2016). Field-aware factorization machines for ctr prediction. In Proceedings of the 10th ACM Conference on Recommender Systems, RecSys \u201916 (pp. 43\u201350). ACM. https:\/\/doi.org\/10.1145\/2959100.2959134","key":"6067_CR14","DOI":"10.1145\/2959100.2959134"},{"unstructured":"Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., & Liu, T. Y. (2017). Lightgbm: A highly efficient gradient boosting decision tree. In NeurIPS.","key":"6067_CR15"},{"unstructured":"Kim, H., Lu, X., Flaxman, S., & Teh, Y. W. (2016). Collaborative filtering with side information: A Gaussian process perspective. Preprint","key":"6067_CR16"},{"unstructured":"Kim, H., & Teh, Y.W. (2018). Scaling up the Automatic Statistician: Scalable Structure Discovery using Gaussian Processes. AISTATS.","key":"6067_CR17"},{"unstructured":"Kim, Y. D., & Choi, S. (2014). Scalable variational Bayesian matrix factorization with side information. In AISTATS.","key":"6067_CR18"},{"unstructured":"Kingma, Diederik & Ba, Jimmy. (2014). Adam: A Method for Stochastic Optimization. International Conference on Learning Representations.","key":"6067_CR19"},{"issue":"3","key":"6067_CR20","doi-asserted-by":"publisher","first-page":"455","DOI":"10.1137\/07070111X","volume":"51","author":"TG Kolda","year":"2009","unstructured":"Kolda, T. G., & Bader, B. W. (2009). Tensor decompositions and applications. SIAM Review, 51(3), 455\u2013500.","journal-title":"SIAM Review"},{"issue":"03","key":"6067_CR21","doi-asserted-by":"publisher","first-page":"280","DOI":"10.1109\/TETC.2014.2330516","volume":"2","author":"L Kuang","year":"2014","unstructured":"Kuang, L., Hao, F., Yang, L. T., Lin, M., Luo, C., & Min, G. (2014). A tensor-based approach for big data representation and dimensionality reduction. IEEE Transactions on Emerging Topics in Computing, 2(03), 280\u2013291. https:\/\/doi.org\/10.1109\/TETC.2014.2330516","journal-title":"IEEE Transactions on Emerging Topics in Computing"},{"doi-asserted-by":"publisher","unstructured":"Liu, T., Wang, Z., Tang, J., Yang, S., Huang, G. Y., & Liu, Z. (2019). Recommender systems with heterogeneous side information (Vol. \u201919, pp. 3027\u20133033). Association for Computing Machinery. https:\/\/doi.org\/10.1145\/3308558.3313580","key":"6067_CR22","DOI":"10.1145\/3308558.3313580"},{"issue":"2","key":"6067_CR23","doi-asserted-by":"publisher","first-page":"298","DOI":"10.1007\/s10618-012-0280-z","volume":"25","author":"A Narita","year":"2012","unstructured":"Narita, A., Hayashi, K., Tomioka, R., & Kashima, H. (2012). Tensor factorization using auxiliary information. Data Mining and Knowledge Discovery, 25(2), 298\u2013324. https:\/\/doi.org\/10.1007\/s10618-012-0280-z","journal-title":"Data Mining and Knowledge Discovery"},{"doi-asserted-by":"publisher","unstructured":"Ong, Victor & Nott, David & Smith, Michael. (2017). Gaussian Variational Approximation With a Factor Covariance Structure. Journal of Computational and Graphical Statistics. 27. https:\/\/doi.org\/10.1080\/10618600.2017.1390472.","key":"6067_CR24","DOI":"10.1080\/10618600.2017.1390472"},{"issue":"5","key":"6067_CR25","doi-asserted-by":"publisher","first-page":"2295","DOI":"10.1137\/090752286","volume":"33","author":"IV Oseledets","year":"2011","unstructured":"Oseledets, I. V. (2011). Tensor-train decomposition. SIAM Journal on Scientific Computing, 33(5), 2295\u20132317. https:\/\/doi.org\/10.1137\/090752286","journal-title":"SIAM Journal on Scientific Computing"},{"doi-asserted-by":"publisher","unstructured":"Pal, B., & Jenamani, M. (2018). Kernelized probabilistic matrix factorization for collaborative filtering: Exploiting projected user and item graph (pp. 437\u2013440). Association for Computing Machinery. https:\/\/doi.org\/10.1145\/3240323.3240402","key":"6067_CR26","DOI":"10.1145\/3240323.3240402"},{"unstructured":"Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L. & Lerer, A. (2017). Automatic Differentiation in PyTorch. NIPS 2017 Workshop on Autodiff.","key":"6067_CR27"},{"unstructured":"Rahimi, A., & Recht, B. (2007). Random features for large-scale kernel machines (Vol. NIPS\u201907, pp. 1177\u20131184). Curran Associates Inc.","key":"6067_CR28"},{"doi-asserted-by":"publisher","unstructured":"Rendle, S. (2010). Factorization machines. In G.I. Webb, B. Liu, C. Zhang, D. Gunopulos, & X. Wu (Eds.) ICDM 2010, The 10th IEEE international conference on data mining, Sydney, Australia, 14\u201317 December 2010 (pp. 995\u20131000). IEEE Computer Society. https:\/\/doi.org\/10.1109\/ICDM.2010.127","key":"6067_CR29","DOI":"10.1109\/ICDM.2010.127"},{"doi-asserted-by":"publisher","unstructured":"Rudin, Cynthia. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence. 1, 206\u2013215. https:\/\/doi.org\/10.1038\/s42256-019-0048-x.","key":"6067_CR30","DOI":"10.1038\/s42256-019-0048-x"},{"unstructured":"Saha, A., Misra, R., Acharya, A., & Ravindran, B. (2017). Scalable variational Bayesian factorization machine. Preprint","key":"6067_CR31"},{"unstructured":"Salakhutdinov, R., & Mnih, A. (2007). Probabilistic matrix factorization. In Proceedings of the 20th international conference on neural information processing systems, NeurIPS (pp. 1257\u20131264). Curran Associates Inc. http:\/\/dl.acm.org\/citation.cfm?id=2981562.2981720","key":"6067_CR32"},{"doi-asserted-by":"publisher","unstructured":"Flunkert, Valentin & Salinas, David & Gasthaus, Jan. (2017). DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks. International Journal of Forecasting. 36. https:\/\/doi.org\/10.1016\/j.ijforecast.2019.07.001.","key":"6067_CR33","DOI":"10.1016\/j.ijforecast.2019.07.001"},{"doi-asserted-by":"crossref","unstructured":"Sch\u00f6lkopf, B., Herbrich, R., & Smola, A. J. (2001). A generalized representer theorem. In COLT\/EuroCOLT.","key":"6067_CR34","DOI":"10.1007\/3-540-44581-1_27"},{"unstructured":"Seeger, M., Salinas, D., & Flunkert, V. (2016). Bayesian intermittent demand forecasting for large inventories. In NeurIPS.","key":"6067_CR35"},{"doi-asserted-by":"crossref","unstructured":"Vovk, V. (2013). Kernel Ridge Regression. Empirical Inference.","key":"6067_CR36","DOI":"10.1007\/978-3-642-41136-6_11"},{"doi-asserted-by":"publisher","unstructured":"Wu, X., Shi, B., Dong, Y., Huang, C., & Chawla, N. V. (2019). Neural tensor factorization for temporal interaction learning (Vol. WSDM \u201919, pp. 537\u2013545). Association for Computing Machinery. https:\/\/doi.org\/10.1145\/3289600.3290998","key":"6067_CR37","DOI":"10.1145\/3289600.3290998"},{"unstructured":"Xu, J., Yao, Y., Tong, H., Tao, X., & Lu, J. (2015). Ice-breaking: Mitigating cold-start recommendation problem by rating comparison. In IJCAI international joint conference on artificial intelligence. 24th international joint conference on artificial intelligence, IJCAI 2015","key":"#cr-split#-6067_CR38.1"},{"unstructured":"conference date: 25-07-2015 through 31-07-2015 (Vol. 2015, pp. 3981-3987).","key":"#cr-split#-6067_CR38.2"},{"unstructured":"Yu, H., Rao, N.S., & Dhillon, I.S. (2016). Temporal Regularized Matrix Factorization for High-dimensional Time Series Prediction. NIPS.","key":"6067_CR39"},{"unstructured":"Yu, R., Bahadori, M. T., & Liu, Y. (2014). Fast multivariate spatio-temporal analysis via low rank tensor learning. In Advances in neural information processing systems (pp. 3491\u20133499).","key":"6067_CR40"},{"unstructured":"Zellner, A. (1986). On Assessing Prior Distributions and Bayesian Regression Analysis With g-Prior Distributions. Basic Bayesian Inference and Decision Techniques: Essays in Honor of Bruno de Finetti","key":"6067_CR41"},{"doi-asserted-by":"publisher","unstructured":"Zhai, Y., Ong, Y., & Tsang, I. W. (2014). The emerging \u201cbig dimensionality\u201d. IEEE Computational Intelligence Magazine, 9(3), 14\u201326. https:\/\/doi.org\/10.1109\/MCI.2014.2326099","key":"6067_CR42","DOI":"10.1109\/MCI.2014.2326099"},{"doi-asserted-by":"crossref","unstructured":"Zhang, J., Shi, X., Zhao, S., & King, I. (2019). STAR-GCN: Stacked and Reconstructed Graph Convolutional Networks for Recommender Systems. IJCAI.","key":"6067_CR43","DOI":"10.24963\/ijcai.2019\/592"},{"doi-asserted-by":"publisher","unstructured":"Zhang, M., Tang, J., Zhang, X., & Xue, X. (2014). Addressing cold start in recommender systems: A semi-supervised co-training algorithm. Association for Computing Machinery. 14, 73\u201382 https:\/\/doi.org\/10.1145\/2600428.2609599","key":"6067_CR44","DOI":"10.1145\/2600428.2609599"},{"issue":"4","key":"6067_CR45","doi-asserted-by":"publisher","first-page":"137","DOI":"10.1109\/MSP.2013.2255334","volume":"30","author":"Q Zhao","year":"2013","unstructured":"Zhao, Q., Zhou, G., Adali, T., Zhang, L., & Cichocki, A. (2013). Kernelization of tensor-based models for multiway data analysis: Processing of multidimensional structured data. IEEE Signal Processing Magazine, 30(4), 137\u2013148. https:\/\/doi.org\/10.1109\/MSP.2013.2255334","journal-title":"IEEE Signal Processing Magazine"},{"unstructured":"Zhao, Q., Zhou, G., Xie, S., Zhang, L., & Cichocki, A. (2016). Tensor ring decomposition. Preprint","key":"6067_CR46"}],"container-title":["Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-021-06067-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10994-021-06067-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10994-021-06067-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,7,5]],"date-time":"2022-07-05T21:06:55Z","timestamp":1657055215000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10994-021-06067-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,9]]},"references-count":47,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2022,7]]}},"alternative-id":["6067"],"URL":"https:\/\/doi.org\/10.1007\/s10994-021-06067-7","relation":{},"ISSN":["0885-6125","1573-0565"],"issn-type":[{"type":"print","value":"0885-6125"},{"type":"electronic","value":"1573-0565"}],"subject":[],"published":{"date-parts":[[2021,11,9]]},"assertion":[{"value":"9 April 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 June 2021","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 September 2021","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 November 2021","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}