{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2023,5,11]],"date-time":"2023-05-11T04:26:42Z","timestamp":1683779202797},"reference-count":26,"publisher":"MIT Press","issue":"6","content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,5,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>We study the problem of hyperparameter tuning in sparse matrix factorization under a Bayesian framework. In prior work, an analytical solution of sparse matrix factorization with Laplace prior was obtained by a variational Bayes method under several approximations. Based on this solution, we propose a novel numerical method of hyperparameter tuning by evaluating the zero point of the normalization factor in a sparse matrix prior. We also verify that our method shows excellent performance for ground-truth sparse matrix reconstruction by comparing it with the widely used algorithm of sparse principal component analysis.<\/jats:p>","DOI":"10.1162\/neco_a_01581","type":"journal-article","created":{"date-parts":[[2023,3,21]],"date-time":"2023-03-21T22:26:11Z","timestamp":1679437571000},"page":"1086-1099","update-policy":"http:\/\/dx.doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":0,"title":["Automatic Hyperparameter Tuning in Sparse Matrix Factorization"],"prefix":"10.1162","volume":"35","author":[{"given":"Ryota","family":"Kawasumi","sequence":"first","affiliation":[{"name":"Department of Mathematics, Graduate School of Science and Engineering, Chuo University, Bunkyo-ku, Tokyo 112-8551, Japan rykawasumi@gmail.com"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Koujin","family":"Takeda","sequence":"additional","affiliation":[{"name":"Department of Mechanical Systems Engineering, Graduate School of Science and Engineering, Ibaraki University, Hitachi, Ibaraki 316-8511, Japan koujin.takeda.kt@vc.ibaraki.ac.jp"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2023,5,12]]},"reference":[{"issue":"11","key":"2023050921091202800_B1","doi-asserted-by":"publisher","first-page":"4311","DOI":"10.1109\/TSP.2006.881199","article-title":"K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation","volume":"54","author":"Aharon","year":"2006","journal-title":"IEEE Trans. Signal Process."},{"key":"2023050921091202800_B2","first-page":"944","article-title":"Estimating lasso risk and noise level","volume-title":"Advances in neural information processing systems, 26","author":"Bayati","year":"2013"},{"key":"2023050921091202800_B3","doi-asserted-by":"crossref","first-page":"434","DOI":"10.1137\/050645506","article-title":"A direct formulation for sparse PCA using semidefinite programming","volume":"49","author":"d'Aspremont","year":"2007","journal-title":"SIAM Rev."},{"key":"2023050921091202800_B4","first-page":"809","article-title":"The degrees of freedom of the lasso for general design matrix","volume":"23","author":"Dossal","year":"2013","journal-title":"Stat. Sin."},{"key":"2023050921091202800_B5","first-page":"2443","article-title":"Method of optimal directions for frame design","volume-title":"Proc. of the IEEE International Conference on Acoustics, Speech, and Signal Processing","author":"Engan","year":"1999"},{"key":"2023050921091202800_B6","first-page":"1957","article-title":"Practical approaches to principal component analysis in the presence of missing values","volume":"11","author":"Ilin","year":"2010","journal-title":"J. Mach. Learn. Res."},{"key":"2023050921091202800_B7","doi-asserted-by":"publisher","first-page":"4228","DOI":"10.1109\/TIT.2016.2556702","article-title":"Phase transitions and sample complexity in Bayes-optimal matrix factorization","volume":"62","author":"Kabashima","year":"2016","journal-title":"IEEE Trans. Inf. Theory"},{"key":"2023050921091202800_B8","doi-asserted-by":"crossref","unstructured":"Kawasumi, R., & Takeda, K. (2018). Approximate method of variational Bayesian matrix factorization\/completion with sparse prior. J. Stat. Mech., 053404.","DOI":"10.1088\/1742-5468\/aabc7d"},{"key":"2023050921091202800_B9","doi-asserted-by":"crossref","unstructured":"Krzakala, F., M\u00e9zard, M., & Zdeborov\u00e0, L. (2013). Phase diagram and approximate message passing for blind calibration and dictionary learning. In Proc. of the IEEE International Symposium on Information Theory (pp. 659\u2013663).","DOI":"10.1109\/ISIT.2013.6620308"},{"key":"2023050921091202800_B10","doi-asserted-by":"crossref","unstructured":"Lesieur, T., Krzakala, F., & Zdeborov\u00e0, L. (2015). Phase transitions in sparse PCA. In Proc. of the IEEE International Symposium on Information Theory (pp. 1635\u20131639).","DOI":"10.1109\/ISIT.2015.7282733"},{"key":"2023050921091202800_B11","doi-asserted-by":"crossref","unstructured":"Lesieur, T., Krzakala, F., & Zdeborov\u00e0, L. (2017). Constrained low-rank matrix estimation: Phase transitions, approximate message passing and applications. J. Stat. Mech., 073403.","DOI":"10.1088\/1742-5468\/aa7284"},{"key":"2023050921091202800_B12","first-page":"689","article-title":"Online dictionary learning for sparse coding","volume-title":"Proc. of the International Conference on Machine Learning","author":"Mairal","year":"2009"},{"key":"2023050921091202800_B13","unstructured":"Matsushita, R., & Tanaka, T. (2013). Low-rank matrix reconstruction and clustering via approximate message passing. In C. J. C.Burges, L.Bottou, M.Welling, Z.Ghahramani, & K. Q.Weinberger (Eds.), Advances in neural information processing systems, 26 (pp. 917\u2013925). Curran."},{"key":"2023050921091202800_B14","doi-asserted-by":"publisher","first-page":"242","DOI":"10.1214\/16-AOS1529","article-title":"Consistent parameter estimation for lasso and approximate message passing","volume":"45","author":"Mousavi","year":"2017","journal-title":"Ann. Stat."},{"key":"2023050921091202800_B15","first-page":"2583","article-title":"Theoretical analysis of Bayesian matrix factorization","volume":"12","author":"Nakajima","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"2023050921091202800_B16","doi-asserted-by":"crossref","unstructured":"Obuchi, T., & Kabashima, Y. (2016). Cross validation in lasso and its acceleration. J. Stat. Mech., 053304.","DOI":"10.1088\/1742-5468\/2016\/05\/053304"},{"issue":"6583","key":"2023050921091202800_B17","doi-asserted-by":"publisher","DOI":"10.1038\/381607a0","article-title":"Emergence of simple-cell receptive field properties by learning a sparse code for natural images","volume":"381","author":"Olshausen","year":"1996","journal-title":"Nature"},{"key":"2023050921091202800_B18","doi-asserted-by":"crossref","unstructured":"Olshausen, B. A., & Field, D. J. (1997). Sparse coding with an overcomplete basis set: A strategy employed by V1?Vis. Res., 37(23), 3311\u20133325.","DOI":"10.1016\/S0042-6989(97)00169-7"},{"key":"2023050921091202800_B19","doi-asserted-by":"publisher","DOI":"10.1209\/0295-5075\/103\/28008","article-title":"Statistical mechanics of dictionary learning","volume":"103","author":"Sakata","year":"2013","journal-title":"EPL"},{"key":"2023050921091202800_B20","doi-asserted-by":"crossref","unstructured":"Salakhutdinov, R., & Mnih, A. (2008). Bayesian probabilistic matrix factorization using Markov chain Monte Carlo. In Proc. of the International Conference on Machine Learning (pp. 880\u2013887).","DOI":"10.1145\/1390156.1390267"},{"key":"2023050921091202800_B21","doi-asserted-by":"crossref","unstructured":"Schulke, C., Schniter, P., & Zdeborov\u00e0, L. (2016, December). Phase diagram of matrix compressed sensing. Phys. Rev. E, 94, 062136.","DOI":"10.1103\/PhysRevE.94.062136"},{"key":"2023050921091202800_B22","unstructured":"The USC-SIPI Image Database: Version 6. (2018). Signal and Image Processing Institute, University of Southern California."},{"key":"2023050921091202800_B23","doi-asserted-by":"publisher","first-page":"791","DOI":"10.1007\/s10463-016-0563-z","article-title":"The degrees of freedom of partly smooth regularizers","volume":"69","author":"Vaiter","year":"2017","journal-title":"Ann. Inst. Stat. Math."},{"key":"2023050921091202800_B24","unstructured":"Wang, S., Zhou, W., Lu, H., Maleki, A., & Mirrokni, V. (2018). Approximate leave- one-out for fast parameter tuning in high dimensions. In Proc. of the International Conference on Machine Learning (pp. 5228\u20135237)."},{"key":"2023050921091202800_B25","doi-asserted-by":"publisher","first-page":"265","DOI":"10.1198\/106186006X113430","article-title":"Sparse principal component analysis","volume":"15","author":"Zou","year":"2006","journal-title":"J. Comput. Graph. Stat."},{"key":"2023050921091202800_B26","doi-asserted-by":"publisher","first-page":"2173","DOI":"10.1214\/009053607000000127","article-title":"On the degrees of freedom of the lasso","volume":"35","author":"Zou","year":"2007","journal-title":"Ann. Statist."}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/35\/6\/1086\/2086347\/neco_a_01581.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/neco\/article-pdf\/35\/6\/1086\/2086347\/neco_a_01581.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,10]],"date-time":"2023-05-10T04:33:05Z","timestamp":1683693185000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/35\/6\/1086\/115248\/Automatic-Hyperparameter-Tuning-in-Sparse-Matrix"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,12]]},"references-count":26,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2023,5,12]]},"published-print":{"date-parts":[[2023,5,12]]}},"URL":"https:\/\/doi.org\/10.1162\/neco_a_01581","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,6]]},"published":{"date-parts":[[2023,5,12]]}}}