{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:44:48Z","timestamp":1760240688841,"version":"build-2065373602"},"reference-count":30,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2019,8,25]],"date-time":"2019-08-25T00:00:00Z","timestamp":1566691200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Finding the size of the dictionary is an open issue in dictionary learning (DL). We propose an algorithm that adapts the size during the learning process by using Information Theoretic Criteria (ITC) specialized to the DL problem. The algorithm is built on top of Approximate K-SVD (AK-SVD) and periodically removes the less used atoms or adds new random atoms, based on ITC evaluations for a small number of candidate sub-dictionaries. Numerical experiments on synthetic data show that our algorithm not only finds the true size with very good accuracy, but is also able to improve the representation error in comparison with AK-SVD knowing the true size.<\/jats:p>","DOI":"10.3390\/a12090178","type":"journal-article","created":{"date-parts":[[2019,8,26]],"date-time":"2019-08-26T04:38:23Z","timestamp":1566794303000},"page":"178","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Adaptive-Size Dictionary Learning Using Information Theoretic Criteria"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4555-1714","authenticated-orcid":false,"given":"Bogdan","family":"Dumitrescu","sequence":"first","affiliation":[{"name":"Department of Automatic Control and Computers, University Politehnica of Bucharest, 313 Spl. Independen\u0163ei, 060042 Bucharest, Romania"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5512-0868","authenticated-orcid":false,"given":"Ciprian Doru","family":"Giurc\u0103neanu","sequence":"additional","affiliation":[{"name":"Department of Statistics, University of Auckland, Auckland 1142, New Zealand"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,8,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1045","DOI":"10.1109\/JPROC.2010.2040551","article-title":"Dictionaries for Sparse Representations Modeling","volume":"98","author":"Rubinstein","year":"2010","journal-title":"Proc. IEEE"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1109\/MSP.2010.939537","article-title":"Dictionary Learning","volume":"28","author":"Tosic","year":"2011","journal-title":"IEEE Signal Proc. Mag."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Dumitrescu, B., and Irofti, P. (2018). Dictionary Learning Algorithms and Applications, Springer.","DOI":"10.1007\/978-3-319-78674-2"},{"key":"ref_4","unstructured":"Agarwal, A., Anandkumar, A., Jain, P., Netrapalli, P., and Tandon, R. (2014, January 13\u201315). Learning sparsely used overcomplete dictionaries. Proceedings of the Conference on Learning Theory, Barcelona, Spain."},{"key":"ref_5","unstructured":"Arora, S., Ge, R., and Moitra, A. (2014, January 13\u201315). New algorithms for learning incoherent and overcomplete dictionaries. Proceedings of the Conference on Learning Theory, Barcelona, Spain."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"6290","DOI":"10.1109\/TIT.2015.2460238","article-title":"When can dictionary learning uniquely recover sparse data from subsamples?","volume":"61","author":"Hillar","year":"2015","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"853","DOI":"10.1109\/TIT.2016.2632162","article-title":"Complete dictionary recovery over the sphere I: Overview and the geometric picture","volume":"63","author":"Sun","year":"2017","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2913","DOI":"10.1109\/TSP.2012.2187203","article-title":"An MDL framework for sparse coding and dictionary learning","volume":"60","author":"Ramirez","year":"2012","journal-title":"IEEE Trans. Signal Proc."},{"key":"ref_9","unstructured":"Schnass, K. (2018). Dictionary learning-from local towards global and adaptive. arXiv."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"631","DOI":"10.1109\/LSP.2012.2209871","article-title":"Stagewise K-SVD to Design Efficient Dictionaries for Sparse Representations","volume":"19","author":"Rusu","year":"2012","journal-title":"IEEE Signal Proc. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"3272","DOI":"10.1109\/TSP.2014.2324994","article-title":"An Adaptive Approach to Learn Overcomplete Dictionaries With Efficient Numbers of Elements","volume":"62","author":"Marsousi","year":"2014","journal-title":"IEEE Trans. Signal Proc."},{"key":"ref_12","unstructured":"Feng, J., Song, L., Yang, X., and Zhang, W. (2009, January 7\u201310). Sub clustering K-SVD: Size variable dictionary learning for sparse representations. Proceedings of the 2009 16th IEEE International Conference on Image Processing (ICIP), Cairo, Egypt."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Mazhar, R., and Gader, P.D. (2008, January 8\u201311). EK-SVD: Optimized Dictionary Design for Sparse Representations. Proceedings of the 2008 19th International Conference on Pattern Recognition, Tampa, FL, USA.","DOI":"10.1109\/ICPR.2008.4761362"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Rao, N., and Porikli, F. (2012, January 25\u201330). A Clustering Approach to Optimize Online Dictionary Learning. Proceedings of the International Conference on Acoustics Speech Signal Proc. (ICASSP), Kyoto, Japan.","DOI":"10.1109\/ICASSP.2012.6288126"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2178","DOI":"10.1109\/TSP.2009.2016257","article-title":"Dictionary Learning for Sparse Approximations with the Majorization Method","volume":"57","author":"Yaghoobi","year":"2009","journal-title":"IEEE Trans. Signal Proc."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1007\/s11265-016-1154-1","article-title":"Towards dictionaries of optimal size: a Bayesian non parametric approach","volume":"90","author":"Dang","year":"2018","journal-title":"J. Signal Proc. Syst."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1109\/TIP.2011.2160072","article-title":"Nonparametric Bayesian Dictionary Learning for Analysis of Noisy and Incomplete Images","volume":"21","author":"Zhou","year":"2012","journal-title":"IEEE Trans. Image Proc."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"5007","DOI":"10.1109\/TIP.2014.2360122","article-title":"Bayesian nonparametric dictionary learning for compressed sensing MRI","volume":"23","author":"Huang","year":"2014","journal-title":"IEEE Trans. Image Proc."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1501","DOI":"10.1109\/TIT.2016.2517006","article-title":"On the minimax risk of dictionary learning","volume":"62","author":"Jung","year":"2016","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1214\/aos\/1176344136","article-title":"Estimating the Dimension of a Model","volume":"6","author":"Schwarz","year":"1978","journal-title":"Ann. Stat."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"759","DOI":"10.1093\/biomet\/asn034","article-title":"Extended Bayesian information criteria for model selection with large model spaces","volume":"95","author":"Chen","year":"2008","journal-title":"Biometrika"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2537","DOI":"10.1109\/18.887861","article-title":"MDL denoising","volume":"46","author":"Rissanen","year":"2000","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Rissanen, J. (2007). Information and Complexity in Statistical Modeling, Springer.","DOI":"10.1007\/978-0-387-68812-1"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"3347","DOI":"10.1109\/TSP.2009.2021633","article-title":"MDL denoising revisited","volume":"57","author":"Roos","year":"2009","journal-title":"IEEE Trans. Signal Proc."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Schipp, B., and Kr\u00e4er, W. (2009). Minimum Description Length Model Selection in Gaussian Regression under Data Constraints. Statistical Inference, Econometric Analysis and Matrix Algebra, Physica-Verlag.","DOI":"10.1007\/978-3-7908-2121-5"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1671","DOI":"10.1016\/j.sigpro.2011.03.015","article-title":"Variable selection in linear regression: Several approaches based on normalized maximum likelihood","volume":"91","author":"Razavi","year":"2011","journal-title":"Signal Process."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1007\/BF02479221","article-title":"Autoregressive model fitting for control","volume":"23","author":"Akaike","year":"1971","journal-title":"Ann. Inst. Stat. Math."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1111\/j.2517-6161.1979.tb01072.x","article-title":"The Determination of the Order of an Autoregression","volume":"41","author":"Hannan","year":"1979","journal-title":"J. R. Stat. Soc. Ser. B (Methodol.)"},{"key":"ref_29","unstructured":"Pati, Y., Rezaiifar, R., and Krishnaprasad, P. (1993, January 1\u20133). Orthogonal Matching Pursuit: Recursive Function Approximation with Applications to Wavelet Decomposition. Proceedings of the 27th Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, USA."},{"key":"ref_30","unstructured":"Rubinstein, R., Zibulevsky, M., and Elad, M. (2008). Efficient Implementation of the K-SVD Algorithm Using Batch Orthogonal Matching Pursuit, Technion University. Technical Report CS-2008-08."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/12\/9\/178\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:13:53Z","timestamp":1760188433000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/12\/9\/178"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,8,25]]},"references-count":30,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2019,9]]}},"alternative-id":["a12090178"],"URL":"https:\/\/doi.org\/10.3390\/a12090178","relation":{},"ISSN":["1999-4893"],"issn-type":[{"type":"electronic","value":"1999-4893"}],"subject":[],"published":{"date-parts":[[2019,8,25]]}}}