{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,15]],"date-time":"2026-10-15T22:55:47Z","timestamp":1792104947635,"version":"4.4.0"},"reference-count":159,"publisher":"Emerald","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2012,1,4]]},"abstract":"<jats:p>Sparse estimation methods are aimed at using or obtaining parsimonious representations of data or models. They were first dedicated to linear variable selection but numerous extensions have now emerged such as structured sparsity or kernel selection. It turns out that many of the related estimation problems can be cast as convex optimization problems by regularizing the empirical risk with appropriate nonsmooth norms. The goal of this monograph is to present from a general perspective optimization tools and techniques dedicated to such sparsity-inducing penalties. We cover proximal methods, block-coordinate descent, reweighted l2-penalized techniques, working-set and homotopy methods, as well as non-convex formulations and extensions, and provide an extensive set of experiments to compare various algorithms from a computational point of view.<\/jats:p>","DOI":"10.1561\/2200000015","type":"journal-article","created":{"date-parts":[[2012,2,24]],"date-time":"2012-02-24T09:35:59Z","timestamp":1330076159000},"page":"1-106","source":"Crossref","is-referenced-by-count":618,"title":["Optimization with Sparsity-Inducing Penalties"],"prefix":"10.1108","volume":"4","author":[{"given":"Bach","family":"Francis","sequence":"first","affiliation":[{"name":"INRIA \u2014 SIERRA Project-Team , Laboratoire d\u2019Informatique de l\u2019Ecole Normale Sup\u00e9rieure, 23, avenue d\u2019Italie, Paris, 75013,","place":["France"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rodolphe","family":"Jenatton","sequence":"additional","affiliation":[{"name":"INRIA \u2014 SIERRA Project-Team"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Julien","family":"Mairal","sequence":"additional","affiliation":[{"name":"Department of Statistics, University of California , Berkeley, 94720-1776,","place":["CA, USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guillaume","family":"Obozinski","sequence":"additional","affiliation":[{"name":"INRIA \u2014 SIERRA Project-Team"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"140","published-online":{"date-parts":[[2012,1,4]]},"reference":[{"key":"2026033014014062100_ref001","first-page":"803","article-title":"A new approach to collaborative filtering: Operator estimation with spectral regularization","volume":"10","author":"Abernethy","year":"(2009)","journal-title":"Journal of Machine Learning Research"},{"key":"2026033014014062100_ref002","first-page":"565","article-title":"Variable sparsity kernel learning","volume":"12","author":"Aflalo","year":"(2011)","journal-title":"Journal of Machine Learning Research"},{"issue":"(11)","key":"2026033014014062100_ref003","doi-asserted-by":"crossref","first-page":"4311","DOI":"10.1109\/TSP.2006.881199","article-title":"K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation","volume":"54","author":"Aharon","year":"(2006)","journal-title":"IEEE Transactions on Signal Processing"},{"key":"2026033014014062100_ref004","doi-asserted-by":"crossref","DOI":"10.1145\/1273496.1273499","article-title":"Uncovering shared structures in multiclass classification","author":"Amit","year":"(2007)","journal-title":"Proceedings of the International Conference on Machine Learning (ICML)"},{"key":"2026033014014062100_ref005","article-title":"Sparse probabilistic projections","author":"Archambeau","year":"(2008)","journal-title":"Advances in Neural Information Processing Systems (NIPS)"},{"issue":"(3)","key":"2026033014014062100_ref006","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1007\/s10994-007-5040-8","article-title":"Convex multi-task feature learning","volume":"73","author":"Argyriou","year":"(2008)","journal-title":"Machine Learning"},{"key":"2026033014014062100_ref007","first-page":"1179","article-title":"Consistency of the group lasso and multiple kernel learning","volume":"9","author":"Bach","year":"(2008)","journal-title":"Journal of Machine Learning Research"},{"key":"2026033014014062100_ref008","first-page":"1019","article-title":"Consistency of trace norm minimization","volume":"9","author":"Bach","year":"(2008)","journal-title":"Journal of Machine Learning Research"},{"key":"2026033014014062100_ref009","article-title":"Exploring large feature spaces with hierarchical multiple kernel learning","author":"Bach","year":"(2008)","journal-title":"Advances in Neural Information Processing Systems (NIPS)"},{"key":"2026033014014062100_ref010","article-title":"Structured sparsity-inducing norms through submodular functions","author":"Bach","year":"(2010)","journal-title":"Advances in Neural Information Processing Systems (NIPS)"},{"key":"2026033014014062100_ref011","article-title":"Shaping level sets with submodular functions","author":"Bach","year":"(2011)","journal-title":"Advances in Neural Information Processing Systems (NIPS)"},{"key":"2026033014014062100_ref012","volume-title":"Optimization for Machine Learning","author":"Bach","year":"(2011)"},{"key":"2026033014014062100_ref013","article-title":"Multiple kernel learning, conic duality, and the SMO algorithm","author":"Bach","year":"(2004)","journal-title":"Proceedings of the International Conference on Machine Learning (ICML)"},{"key":"2026033014014062100_ref014","article-title":"Convex sparse matrix factorizations","author":"Bach","year":"(2008)","journal-title":"Preprint arXiv:0812.1869v1"},{"issue":"(4)","key":"2026033014014062100_ref015","doi-asserted-by":"crossref","first-page":"1982","DOI":"10.1109\/TIT.2010.2040894","article-title":"Model-based compressive sensing","volume":"56","author":"Baraniuk","year":"(2010)","journal-title":"IEEE Transactions on Information Theory"},{"issue":"(1)","key":"2026033014014062100_ref016","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1007\/BF01386026","article-title":"Absolute and monotonic norms","volume":"3","author":"Bauer","year":"(1961)","journal-title":"Numerische Mathematik"},{"issue":"(1)","key":"2026033014014062100_ref017","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1137\/080716542","article-title":"A fast iterative shrinkage-thresholding algorithm for linear inverse problems","volume":"2","author":"Beck","year":"(2009)","journal-title":"SIAM Journal on Imaging Sciences"},{"key":"2026033014014062100_ref018","volume-title":"Nonlinear Programming","author":"Bertsekas","year":"(1999)"},{"issue":"(4)","key":"2026033014014062100_ref019","first-page":"1705","article-title":"Simultaneous analysis of Lasso and Dantzig selector","volume":"37","author":"Bickel","year":"(2009)","journal-title":"Annals of Statistics"},{"key":"2026033014014062100_ref020","doi-asserted-by":"crossref","DOI":"10.1007\/978-0-387-31256-9","volume-title":"Convex Analysis and Nonlinear Optimization: Theory and Examples","author":"Borwein","year":"(2006)"},{"key":"2026033014014062100_ref021","article-title":"Online algorithms and stochastic approximations","volume":"5","author":"Bottou","year":"(1998)","journal-title":"Online Learning and Neural Networks"},{"key":"2026033014014062100_ref022","article-title":"The tradeoffs of large scale learning","author":"Bottou","year":"(2008)","journal-title":"Advances in Neural Information Processing Systems (NIPS)"},{"key":"2026033014014062100_ref023","article-title":"Large scale online learning","author":"Bottou","year":"(2004)","journal-title":"Advances in Neural Information Processing Systems (NIPS)"},{"issue":"(1)","key":"2026033014014062100_ref024","first-page":"1","article-title":"Distributed optimization and statistical learning via the alternating direction method of multipliers","volume":"3","author":"Boyd","year":"(2011)","journal-title":"Foundations and Trends in Machine Learning"},{"key":"2026033014014062100_ref025","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511804441","volume-title":"Convex Optimization","author":"Boyd","year":"(2004)"},{"key":"2026033014014062100_ref026","article-title":"Convex coding","author":"Bradley","year":"(2009)","journal-title":"Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence (UAI)"},{"issue":"(3)","key":"2026033014014062100_ref027","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1016\/0167-6377(84)90010-5","article-title":"An O(n) algorithm for quadratic knapsack problems","volume":"3","author":"Brucker","year":"(1984)","journal-title":"Operations Research Letters"},{"issue":"(4)","key":"2026033014014062100_ref028","first-page":"275","article-title":"Dimension reduction: A guided tour","volume":"2","author":"Burges","year":"(2009)","journal-title":"Machine Learning"},{"key":"2026033014014062100_ref029","doi-asserted-by":"crossref","first-page":"1956","DOI":"10.1137\/080738970","article-title":"A singular value thresholding algorithm for matrix completion","volume":"20","author":"Cai","year":"(2010)","journal-title":"SIAM Journal on Optimization"},{"issue":"(5)","key":"2026033014014062100_ref030","doi-asserted-by":"crossref","first-page":"877","DOI":"10.1007\/s00041-008-9045-x","article-title":"Enhancing sparsity by reweighted L1 minimization","volume":"14","author":"Cand\u00e8s","year":"(2008)","journal-title":"Journal of Fourier Analysis and Applications"},{"key":"2026033014014062100_ref031","doi-asserted-by":"crossref","DOI":"10.1145\/1390156.1390168","article-title":"Sparse Bayesian nonparametric regression","author":"Caron","year":"(2008)","journal-title":"Proceedings of the International Conference on Machine Learning (ICML)"},{"key":"2026033014014062100_ref032","article-title":"Sparse signal recovery using markov random fields","author":"Cehver","year":"(2008)","journal-title":"Advances in Neural Information Processing Systems (NIPS)"},{"key":"2026033014014062100_ref033","article-title":"Total variation minimization and a class of binary MRF models","author":"Chambolle","year":"(2005)","journal-title":"Proceedings of the fifth International Workshop on Energy Minimization Methods in Computer Vision and Pattern Recognition"},{"issue":"(3)","key":"2026033014014062100_ref034","doi-asserted-by":"crossref","first-page":"288","DOI":"10.1007\/s11263-009-0238-9","article-title":"On total variation minimization and surface evolution using parametric maximum flows","volume":"84","author":"Chambolle","year":"(2009)","journal-title":"International Journal of Computer Vision"},{"key":"2026033014014062100_ref035","article-title":"The convex geometry of linear inverse problems","author":"Chandrasekaran","year":"(2010)","journal-title":"Preprint arXiv:1012.0621"},{"issue":"(2)","key":"2026033014014062100_ref036","doi-asserted-by":"crossref","first-page":"421","DOI":"10.1137\/S1052623495290179","article-title":"Convergence rates in forward-backward splitting","volume":"7","author":"Chen","year":"(1997)","journal-title":"SIAM Journal on Optimization"},{"key":"2026033014014062100_ref037","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1137\/S1064827596304010","article-title":"Atomic decomposition by basis pursuit","volume":"20","author":"Chen","year":"(1999)","journal-title":"SIAM Journal on Scientific Computing"},{"key":"2026033014014062100_ref038","author":"Combettes","year":"(2011)"},{"issue":"(4)","key":"2026033014014062100_ref039","first-page":"1168","article-title":"Signal recovery by proximal forward-backward splitting","volume":"4","author":"Combettes","year":"(2006)","journal-title":"SIAM Multiscale Modeling and Simulation"},{"key":"2026033014014062100_ref040","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1049\/ip-vis:19990445","article-title":"Forward sequential algorithms for best basis selection","author":"Cotter","year":"(1999)","journal-title":"IEEE Proceedings of Vision Image and Signal Processing"},{"issue":"(1)","key":"2026033014014062100_ref041","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1002\/cpa.20303","article-title":"Iteratively reweighted least squares minimization for sparse recovery","volume":"63","author":"Daubechies","year":"(2010)","journal-title":"Communications on Pure and Applied Mathematics"},{"issue":"(432)","key":"2026033014014062100_ref042","doi-asserted-by":"crossref","first-page":"1200","DOI":"10.1080\/01621459.1995.10476626","article-title":"Adapting to unknown smoothness via wavelet shrinkage","volume":"90","author":"Donoho","year":"(1995)","journal-title":"Journal of the American Statistical Association"},{"key":"2026033014014062100_ref043","first-page":"2121","article-title":"Adaptive subgradient methods for online learning and stochastic optimization","volume":"12","author":"Duchi","year":"(2011)","journal-title":"Journal of Machine Learning Research"},{"issue":"(2)","key":"2026033014014062100_ref044","first-page":"407","article-title":"Least angle regression","volume":"32","author":"Efron","year":"(2004)","journal-title":"Annals of Statistics"},{"issue":"(12)","key":"2026033014014062100_ref045","doi-asserted-by":"crossref","first-page":"3736","DOI":"10.1109\/TIP.2006.881969","article-title":"Image denoising via sparse and redundant representations over learned dictionaries","volume":"15","author":"Elad","year":"(2006)","journal-title":"IEEE Transactions on Image Processing"},{"key":"2026033014014062100_ref046","article-title":"Method of optimal directions for frame design","author":"Engan","year":"(1999)","journal-title":"Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP)"},{"issue":"(456)","key":"2026033014014062100_ref047","doi-asserted-by":"crossref","first-page":"1348","DOI":"10.1198\/016214501753382273","article-title":"Variable selection via nonconcave penalized likelihood and its oracle properties","volume":"96","author":"Fan","year":"(2001)","journal-title":"Journal of the American Statistical Association"},{"key":"2026033014014062100_ref048","doi-asserted-by":"crossref","first-page":"4734","DOI":"10.1109\/ACC.2001.945730","article-title":"A rank minimization heuristic with application to minimum order system approximation","volume":"6","author":"Fazel","year":"(2001)","journal-title":"Proceedings of the American Control Conference"},{"key":"2026033014014062100_ref049","article-title":"A note on the group lasso and a sparse group lasso","author":"Friedman","year":"(2010)","journal-title":"Preprint arXiv:1001.0736v1"},{"issue":"(5)","key":"2026033014014062100_ref050","first-page":"1189","article-title":"Greedy function approximation: a gradient boosting machine","volume":"29","author":"Friedman","year":"(2001)","journal-title":"Annals of Statistics"},{"issue":"(3)","key":"2026033014014062100_ref051","doi-asserted-by":"crossref","first-page":"397","DOI":"10.1080\/10618600.1998.10474784","article-title":"Penalized regressions: The bridge versus the lasso","volume":"7","author":"Fu","year":"(1998)","journal-title":"Journal of Computational and Graphical Statistics"},{"issue":"(12)","key":"2026033014014062100_ref052","doi-asserted-by":"crossref","first-page":"4686","DOI":"10.1109\/TSP.2009.2026004","article-title":"Recovering sparse signals with non-convex penalties and DC programming","volume":"57","author":"Gasso","year":"(2009)","journal-title":"IEEE Transactions on Signal Processing"},{"issue":"(3)","key":"2026033014014062100_ref053","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1198\/004017007000000245","article-title":"Large-scale bayesian logistic regression for text categorization","volume":"49","author":"Genkin","year":"(2007)","journal-title":"Technometrics"},{"key":"2026033014014062100_ref054","doi-asserted-by":"crossref","DOI":"10.1137\/1.9781611970838","volume-title":"Augmented Lagrangian and Operator-splitting Methods in Nonlinear Mechanics","author":"Glowinski","year":"(1989)"},{"key":"2026033014014062100_ref055","article-title":"Outcomes of the equivalence of adaptive ridge with least absolute shrinkage","author":"Grandvalet","year":"(1999)","journal-title":"Advances in Neural Information Processing Systems (NIPS)"},{"key":"2026033014014062100_ref056","article-title":"Compressible distributions for high-dimensional statistics","author":"Gribonval","year":"(2011)","journal-title":"preprint arXiv:1102.1249v2"},{"key":"2026033014014062100_ref057","author":"Harchaoui","year":"(2008)"},{"key":"2026033014014062100_ref058","article-title":"Catching change-points with Lasso","author":"Harchaoui","year":"(2008)","journal-title":"Advances in Neural Information Processing Systems (NIPS)"},{"key":"2026033014014062100_ref059","article-title":"Sparse approximation via iterative thresholding","author":"Herrity","year":"(2006)","journal-title":"Proceedings of the International Conference on Acoustics, Speech and Signal Processing (ICASSP)"},{"key":"2026033014014062100_ref060","article-title":"Accelerated gradient methods for stochastic optimization and online learning","author":"Hu","year":"(2009)","journal-title":"Advances in Neural Information Processing Systems (NIPS)"},{"issue":"(4)","key":"2026033014014062100_ref061","first-page":"1978","article-title":"The benefit of group sparsity","volume":"38","author":"Huang","year":"(2010)","journal-title":"Annals of Statistics"},{"key":"2026033014014062100_ref062","article-title":"Learning with structured sparsity","author":"Huang","year":"(2009)","journal-title":"Proceedings of the International Conference on Machine Learning (ICML)"},{"issue":"(2)","key":"2026033014014062100_ref063","first-page":"730","article-title":"Spike and slab variable selection: frequentist and Bayesian strategies","volume":"33","author":"Ishwaran","year":"(2005)","journal-title":"Annals of Statistics"},{"key":"2026033014014062100_ref064","article-title":"Group Lasso with overlaps and graph Lasso","author":"Jacob","year":"(2009)","journal-title":"Proceedings of the International Conference on Machine Learning (ICML)"},{"key":"2026033014014062100_ref065","author":"Jenatton","year":"(2012)"},{"key":"2026033014014062100_ref066","first-page":"2777","article-title":"Structured variable selection with sparsity-inducing norms","volume":"12","author":"Jenatton","year":"(2011)","journal-title":"Journal of Machine Learning Research"},{"key":"2026033014014062100_ref067","article-title":"Multi-scale mining of fMRI data with hierarchical structured sparsity","author":"Jenatton","year":"(2011)","journal-title":"International Workshop on Pattern Recognition in Neuroimaging (PRNI)"},{"key":"2026033014014062100_ref068","article-title":"Proximal methods for sparse hierarchical dictionary learning","author":"Jenatton","year":"(2010)","journal-title":"Proceedings of the International Conference on Machine Learning (ICML)"},{"key":"2026033014014062100_ref069","first-page":"2297","article-title":"Proximal methods for hierarchical sparse coding","volume":"12","author":"Jenatton","year":"(2011)","journal-title":"Journal of Machine Learning Research"},{"key":"2026033014014062100_ref070","article-title":"Structured sparse principal component analysis","author":"Jenatton","year":"(2010)","journal-title":"Proceedings of International Workshop on Artificial Intelligence and Statistics (AISTATS)"},{"issue":"(3)","key":"2026033014014062100_ref071","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1007\/BF02289588","article-title":"Hierarchical clustering schemes","volume":"32","author":"Johnson","year":"(1967)","journal-title":"Psychometrika"},{"key":"2026033014014062100_ref072","article-title":"Learning invariant features through topographic filter maps","author":"Kavukcuoglu","year":"(2009)","journal-title":"Proceedings of IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"2026033014014062100_ref073","article-title":"Tree-guided group lasso for multi-task regression with structured sparsity","author":"Kim","year":"(2010)","journal-title":"Proceedings of the International Conference on Machine Learning (ICML)"},{"key":"2026033014014062100_ref074","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/0022-247X(71)90184-3","article-title":"Some results on Tchebycheffian spline functions","volume":"33","author":"Kimeldorf","year":"(1971)","journal-title":"Journal of Mathematical Analysis and Applications"},{"issue":"(8)","key":"2026033014014062100_ref075","first-page":"1519","article-title":"An interior-point method for large-scale l1-regularized logistic regression","volume":"8","author":"Koh","year":"(2007)","journal-title":"Journal of Machine Learning Research"},{"issue":"(6)","key":"2026033014014062100_ref076","doi-asserted-by":"crossref","first-page":"957","DOI":"10.1109\/TPAMI.2005.127","article-title":"Sparse multinomial logistic regression: Fast algorithms and generalization bounds","volume":"27","author":"Krishnapuram","year":"(2005)","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"2026033014014062100_ref077","first-page":"27","article-title":"Learning the kernel matrix with semidefinite programming","volume":"5","author":"Lanckriet","year":"(2004)","journal-title":"Journal of Machine Learning Research"},{"key":"2026033014014062100_ref078","doi-asserted-by":"crossref","first-page":"2626","DOI":"10.1093\/bioinformatics\/bth294","article-title":"A statistical framework for genomic data fusion","volume":"20","author":"Lanckriet","year":"(2004)","journal-title":"Bioinformatics"},{"key":"2026033014014062100_ref079","article-title":"Efficient sparse coding algorithms","author":"Lee","year":"(2007)","journal-title":"Advances in Neural Information Processing Systems (NIPS)"},{"key":"2026033014014062100_ref080","article-title":"Itakura-Saito nonnegative matrix factorization with group sparsity","author":"Lef\u00e8vre","year":"(2011)","journal-title":"Proceedings of the International Conference on Acoustics, Speech and Signal Processing (ICASSP)"},{"issue":"(6)","key":"2026033014014062100_ref081","doi-asserted-by":"crossref","first-page":"964","DOI":"10.1137\/0716071","article-title":"Splitting algorithms for the sum of two nonlinear operators","volume":"16","author":"Lions","year":"(1979)","journal-title":"SIAM Journal on Numerical Analysis"},{"key":"2026033014014062100_ref082","article-title":"Blockwise coordinate descent procedures for the multi-task lasso, with applications to neural semantic basis discovery","author":"Liu","year":"(2009)","journal-title":"Proceedings of the International Conference on Machine Learning (ICML)"},{"key":"2026033014014062100_ref083","article-title":"Taking advantage of sparsity in multi-task learning","author":"Lounici","year":"(2009)","journal-title":"Preprint arXiv:0903.1468"},{"issue":"(4)","key":"2026033014014062100_ref084","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1016\/0167-6377(89)90064-3","article-title":"A linear-time median-finding algorithm for projecting a vector on the simplex of Rn","volume":"8","author":"Maculan","year":"(1989)","journal-title":"Operations Research Letters"},{"key":"2026033014014062100_ref085","author":"Mairal","year":"(2010)"},{"key":"2026033014014062100_ref086","first-page":"19","article-title":"Online learning for matrix factorization and sparse coding","volume":"11","author":"Mairal","year":"(2010)","journal-title":"Journal of Machine Learning Research"},{"key":"2026033014014062100_ref087","article-title":"Network flow algorithms for structured sparsity","author":"Mairal","year":"(2010)","journal-title":"Advances in Neural Information Processing Systems (NIPS)"},{"key":"2026033014014062100_ref088","first-page":"2681","article-title":"Convex and network flow optimization for structured sparsity","volume":"12","author":"Mairal","year":"(2011)","journal-title":"Journal of Machine Learning Research"},{"issue":"(12)","key":"2026033014014062100_ref089","doi-asserted-by":"crossref","first-page":"3397","DOI":"10.1109\/78.258082","article-title":"Matching pursuit in a time-frequency dictionary","volume":"41","author":"Mallat","year":"(1993)","journal-title":"IEEE Transactions on Signal Processing"},{"issue":"(1)","key":"2026033014014062100_ref090","first-page":"77","article-title":"Portfolio selection","volume":"7","author":"Markowitz","year":"(1952)","journal-title":"Journal of Finance"},{"key":"2026033014014062100_ref091","article-title":"R\u00e8gularisation d\u2019in\u00e8quations variationnelles par approximations successives","author":"Martinet","year":"(1970)","journal-title":"Revue fran\u00e7aise d\u2019informatique et de recherche op\u00e8rationnelle, s\u00e8rie rouge"},{"key":"2026033014014062100_ref092","article-title":"Structured sparsity in structured prediction","author":"Martins","year":"(2011)","journal-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)"},{"key":"2026033014014062100_ref093","article-title":"Regularizers for structured sparsity","author":"Micchelli","year":"(2011)","journal-title":"Preprint arXiv:1010.0556v2"},{"key":"2026033014014062100_ref094","first-page":"2897","article-title":"Fonctions convexes duales et points proximaux dans un espace hilbertien","volume":"255","author":"Moreau","year":"(1962)","journal-title":"Comptes-Rendus de l\u2019Acad\u00e8mie des Sciences de Paris, S\u00e8rie A, Math\u00e8matiques"},{"key":"2026033014014062100_ref095","first-page":"418","article-title":"Solving structured sparsity regularization with proximal methods","author":"Mosci","year":"(2010)","journal-title":"Machine Learning and Knowledge Discovery in Databases"},{"key":"2026033014014062100_ref096","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1137\/S0097539792240406","article-title":"Sparse approximate solutions to linear systems","volume":"24","author":"Natarajan","year":"(1995)","journal-title":"SIAM Journal on Computing"},{"key":"2026033014014062100_ref097","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4612-0745-0","volume-title":"Bayesian Learning for Neural Networks","author":"Neal","year":"(1996)"},{"issue":"(3)","key":"2026033014014062100_ref098","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1016\/j.acha.2008.07.002","article-title":"CoSaMP: Iterative signal recovery from incomplete and inaccurate samples","volume":"26","author":"Needell","year":"(2009)","journal-title":"Applied and Computational Harmonic Analysis"},{"key":"2026033014014062100_ref099","article-title":"A unified framework for high-dimensional analysis of M-estimators with decomposable regularizers","author":"Negahban","year":"(2009)","journal-title":"Advances in Neural Information Processing Systems (NIPS)"},{"key":"2026033014014062100_ref100","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4419-8853-9","volume-title":"Introductory Lectures on Convex Optimization: A Basic Course","author":"Nesterov","year":"(2004)"},{"issue":"(1)","key":"2026033014014062100_ref101","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1007\/s10107-004-0552-5","article-title":"Smooth minimization of non-smooth functions","volume":"103","author":"Nesterov","year":"(2005)","journal-title":"Mathematical Programming"},{"key":"2026033014014062100_ref102","article-title":"Gradient methods for minimizing composite objective function","author":"Nesterov","year":"(2007)","journal-title":"Technical Report, Center for Operations Research and Econometrics (CORE), Catholic University of Louvain, Discussion paper"},{"key":"2026033014014062100_ref103","article-title":"Efficiency of coordinate descent methods on huge-scale optimization problems","author":"Nesterov","year":"(2010)","journal-title":"Technical Report, Center for Operations Research and Econometrics (CORE), Catholic University of Louvain, Discussion paper"},{"key":"2026033014014062100_ref104","volume-title":"Numerical Optimization","author":"Nocedal","year":"(2006)"},{"key":"2026033014014062100_ref105","article-title":"Group Lasso with overlaps: the Latent Group Lasso approach","author":"Obozinski","year":"(2011)","journal-title":"preprint HAL \u2014 inria\u201300628498"},{"issue":"(2)","key":"2026033014014062100_ref106","first-page":"231","article-title":"Joint covariate selection and joint subspace selection for multiple classification problems","volume":"20","author":"Obozinski","year":"(2009)","journal-title":"Statistics and Computing"},{"key":"2026033014014062100_ref107","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1038\/381607a0","article-title":"Emergence of simple-cell receptive field properties by learning a sparse code for natural images","volume":"381","author":"Olshausen","year":"(1996)","journal-title":"Nature"},{"issue":"(2)","key":"2026033014014062100_ref108","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1080\/10618600.2000.10474883","article-title":"On the Lasso and its dual","volume":"9","author":"Osborne","year":"(2000)","journal-title":"Journal of Computational and Graphical Statistics"},{"key":"2026033014014062100_ref109","article-title":"Multi-task feature learning","author":"Pontil","year":"(2007)","journal-title":"Advances in Neural Information Processing Systems (NIPS)"},{"key":"2026033014014062100_ref110","first-page":"2491","article-title":"SimpleMKL","volume":"9","author":"Rakotomamonjy","year":"(2008)","journal-title":"Journal of Machine Learning Research"},{"key":"2026033014014062100_ref111","article-title":"Convex approaches to model wavelet sparsity patterns","author":"Rao","year":"(2011)","journal-title":"International Conference on Image Processing (ICIP)"},{"issue":"(13)","key":"2026033014014062100_ref112","doi-asserted-by":"crossref","first-page":"375","DOI":"10.1093\/bioinformatics\/btn188","article-title":"Classification of arrayCGH data using fused SVM","volume":"24","author":"Rapaport","year":"(2008)","journal-title":"Bioinformatics"},{"key":"2026033014014062100_ref113","doi-asserted-by":"crossref","first-page":"1009","DOI":"10.1111\/j.1467-9868.2009.00718.x","article-title":"Sparse additive models","volume":"71","author":"Ravikumar","year":"(2009)","journal-title":"Journal of the Royal Statistical Society. Series B, statistical methodology"},{"issue":"(4)","key":"2026033014014062100_ref114","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1007\/BF01920852","article-title":"Ein verfahren zur l\u00f6sung parameterabh\u00e4ngiger, nichtlinearer maximum-probleme","volume":"6","author":"Ritter","year":"(1962)","journal-title":"Mathematical Methods of Operations Research"},{"key":"2026033014014062100_ref115","volume-title":"Convex Analysis","author":"Rockafellar","year":"(1997)"},{"key":"2026033014014062100_ref116","doi-asserted-by":"crossref","DOI":"10.1145\/1390156.1390263","article-title":"The Group-Lasso for generalized linear models: uniqueness of solutions and efficient algorithms","author":"Roth","year":"(2008)","journal-title":"Proceedings of the International Conference on Machine Learning (ICML)"},{"issue":"(1-4)","key":"2026033014014062100_ref117","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1016\/0167-2789(92)90242-F","article-title":"Nonlinear total variation based noise removal algorithms","volume":"60","author":"Rudin","year":"(1992)","journal-title":"Physica D: Nonlinear Phenomena"},{"key":"2026033014014062100_ref118","article-title":"Fast optimization methods for L1 regularization: A comparative study and two new approaches","author":"Schmidt","year":"(2007)","journal-title":"Proceedings of the European Conference on Machine Learning (ECML)"},{"key":"2026033014014062100_ref119","volume-title":"Optimization for Machine Learning","author":"Schmidt","year":"(2011)"},{"key":"2026033014014062100_ref120","article-title":"Convergence rates of inexact proximal-gradient methods for convex optimization","author":"Schmidt","year":"(2011)","journal-title":"Advances in Neural Information Processing Systems (NIPS)"},{"key":"2026033014014062100_ref121","article-title":"Convex structure learning in log-linear models: Beyond pairwise potentials","author":"Schmidt","year":"(2010)","journal-title":"Proceedings of International Workshop on Artificial Intelligence and Statistics (AISTATS)"},{"key":"2026033014014062100_ref122","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/4175.001.0001","volume-title":"Learning with Kernels","author":"Sch\u00f6lkopf","year":"(2001)"},{"key":"2026033014014062100_ref123","first-page":"759","article-title":"Bayesian inference and optimal design for the sparse linear model","volume":"9","author":"Seeger","year":"(2008)","journal-title":"Journal of Machine Learning Research"},{"key":"2026033014014062100_ref124","article-title":"Stochastic methods for l1-regularized loss minimization","author":"Shalev-Shwartz","year":"(2009)","journal-title":"Proceedings of the International Conference on Machine Learning (ICML)"},{"key":"2026033014014062100_ref125","doi-asserted-by":"crossref","DOI":"10.1137\/1.9780898718751","volume-title":"Lectures on Stochastic Programming: Modeling and Theory","author":"Shapiro","year":"(2009)"},{"key":"2026033014014062100_ref126","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511809682","volume-title":"Kernel Methods for Pattern Analysis","author":"Shawe-Taylor","year":"(2004)"},{"issue":"(17)","key":"2026033014014062100_ref127","doi-asserted-by":"crossref","first-page":"2246","DOI":"10.1093\/bioinformatics\/btg308","article-title":"A simple and efficient algorithm for gene selection using sparse logistic regression","volume":"19","author":"Shevade","year":"(2003)","journal-title":"Bioinformatics"},{"key":"2026033014014062100_ref128","first-page":"1531","article-title":"Large scale multiple kernel learning","volume":"7","author":"Sonnenburg","year":"(2006)","journal-title":"Journal of Machine Learning Research"},{"issue":"(9)","key":"2026033014014062100_ref129","doi-asserted-by":"crossref","first-page":"4183","DOI":"10.1109\/TSP.2011.2157912","article-title":"C-HiLasso: A collaborative hierarchical sparse modeling framework","volume":"59","author":"Sprechmann","year":"(2011)","journal-title":"IEEE Transactions on Signal Processing"},{"key":"2026033014014062100_ref130","article-title":"Maximum-margin matrix factorization","author":"Srebro","year":"(2005)","journal-title":"Advances in Neural Information Processing Systems (NIPS)"},{"key":"2026033014014062100_ref131","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1007\/s10994-011-5252-9","article-title":"SpicyMKL: A fast algorithm for multiple kernel learning with thousands of kernels","volume":"85","author":"Suzuki","year":"(2011)","journal-title":"Machine Learning"},{"key":"2026033014014062100_ref132","article-title":"Hierarchical penalization","author":"Szafranski","year":"(2007)","journal-title":"Advances in Neural Information Processing Systems (NIPS)"},{"issue":"(1)","key":"2026033014014062100_ref133","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression shrinkage and selection via the Lasso","volume":"58","author":"Tibshirani","year":"(1996)","journal-title":"Journal of the Royal Statistical Society Series B"},{"issue":"(1)","key":"2026033014014062100_ref134","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1111\/j.1467-9868.2005.00490.x","article-title":"Sparsity and smoothness via the fused Lasso","volume":"67","author":"Tibshirani","year":"(2005)","journal-title":"Journal of the Royal Statistical Society Series B"},{"key":"2026033014014062100_ref135","volume-title":"Optimization for Machine Learning","author":"Tomioka","year":"(2011)"},{"issue":"(10)","key":"2026033014014062100_ref136","first-page":"2231","article-title":"Greed is good: Algorithmic results for sparse approximation","volume":"50","author":"Tropp","year":"(2004)","journal-title":"IEEE Transactions on Signal Processing"},{"key":"2026033014014062100_ref137","first-page":"572","article-title":"Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit","volume":"86","author":"Tropp","year":"(2006)","journal-title":"Signal Processing, Special Issue \u201cSparse Approximations in Signal and Image Processing\u201d"},{"key":"2026033014014062100_ref138","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1137\/0329006","article-title":"Applications of a splitting algorithm to decomposition in convex programming and variational inequalities","volume":"29","author":"Tseng","year":"(1991)","journal-title":"SIAM Journal on Control and Optimization"},{"key":"2026033014014062100_ref139","article-title":"On accelerated proximal gradient methods for convex-concave optimization","author":"Tseng","year":"(2008)","journal-title":"submitted to SIAM Journal on Optimization"},{"issue":"(1)","key":"2026033014014062100_ref140","doi-asserted-by":"crossref","first-page":"387","DOI":"10.1007\/s10107-007-0170-0","article-title":"A coordinate gradient descent method for nonsmooth separable minimization","volume":"117","author":"Tseng","year":"(2009)","journal-title":"Mathematical Programming"},{"issue":"(3)","key":"2026033014014062100_ref141","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1198\/004017005000000139","article-title":"Simultaneous variable selection","volume":"47","author":"Turlach","year":"(2005)","journal-title":"Technometrics"},{"key":"2026033014014062100_ref142","article-title":"Group sparsity via linear-time projections","author":"Van den Berg","year":"(2008)","journal-title":"Technical report, University of British Columbia, Technical Report number TR-2008-09"},{"key":"2026033014014062100_ref143","article-title":"Sparse structured dictionary learning for brain resting-state activity modeling","author":"Varoquaux","year":"(2010)","journal-title":"NIPS Workshop on Practical Applications of Sparse Modeling: Open Issues and New Directions"},{"key":"2026033014014062100_ref144","article-title":"Fast detection of multiple change-points shared by many signals using group LARS","author":"Vert","year":"(2010)","journal-title":"Advances in Neural Information Processing Systems (NIPS)"},{"issue":"(5)","key":"2026033014014062100_ref145","doi-asserted-by":"crossref","first-page":"2183","DOI":"10.1109\/TIT.2009.2016018","article-title":"Sharp thresholds for noisy and high-dimensional recovery of sparsity using l1-constrained quadratic programming","volume":"55","author":"Wainwright","year":"(2009)","journal-title":"IEEE Transactions on Information Theory"},{"key":"2026033014014062100_ref146","volume-title":"Applied Linear Regression","author":"Weisberg","year":"(1980)"},{"key":"2026033014014062100_ref147","article-title":"A new view of automatic relevance determination","author":"Wipf","year":"(2008)","journal-title":"Advances in Neural Information Processing Systems (NIPS)"},{"issue":"(8)","key":"2026033014014062100_ref148","first-page":"2153","article-title":"Sparse bayesian learning for basis selection","volume":"52","author":"Wipf","year":"(2004)"},{"issue":"(2)","key":"2026033014014062100_ref149","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1109\/TPAMI.2008.79","article-title":"Robust face recognition via sparse representation","volume":"31","author":"Wright","year":"(2009)","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"2026033014014062100_ref150","article-title":"Accelerated block-coordinate relaxation for regularized optimization","author":"Wright","year":"(2010)","journal-title":"Technical report, University of Wisconsin-Madison"},{"issue":"(7)","key":"2026033014014062100_ref151","doi-asserted-by":"crossref","first-page":"2479","DOI":"10.1109\/TSP.2009.2016892","article-title":"Sparse reconstruction by separable approximation","volume":"57","author":"Wright","year":"(2009)","journal-title":"IEEE Transactions on Signal Processing"},{"issue":"(1)","key":"2026033014014062100_ref152","first-page":"224","article-title":"Coordinate descent algorithms for lasso penalized regression","volume":"2","author":"Wu","year":"(2008)","journal-title":"Annals of Applied Statistics"},{"key":"2026033014014062100_ref153","first-page":"2543","article-title":"Dual averaging methods for regularized stochastic learning and online optimization","volume":"9","author":"Xiao","year":"(2010)","journal-title":"Journal of Machine Learning Research"},{"key":"2026033014014062100_ref154","article-title":"Robust PCA via outlier pursuit","author":"Xu","year":"(2010)","journal-title":"Preprint arXiv:1010.4237"},{"key":"2026033014014062100_ref155","article-title":"A comparison of optimization methods for large-scale l1-regularized linear classification","author":"Yuan","year":"(2010)","journal-title":"Technical Report, Department of Computer Science, National University of Taiwan"},{"key":"2026033014014062100_ref156","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1111\/j.1467-9868.2005.00532.x","article-title":"Model selection and estimation in regression with grouped variables","volume":"68","author":"Yuan","year":"(2006)","journal-title":"Journal of the Royal Statistical Society Series B"},{"issue":"(6A)","key":"2026033014014062100_ref157","first-page":"3468","article-title":"The composite absolute penalties family for grouped and hierarchical variable selection","volume":"37","author":"Zhao","year":"(2009)","journal-title":"Annals of Statistics"},{"key":"2026033014014062100_ref158","first-page":"2541","article-title":"On model selection consistency of Lasso","volume":"7","author":"Zhao","year":"(2006)","journal-title":"Journal of Machine Learning Research"},{"issue":"(2)","key":"2026033014014062100_ref159","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1111\/j.1467-9868.2005.00503.x","article-title":"Regularization and variable selection via the elastic net","volume":"67","author":"Zou","year":"(2005)","journal-title":"Journal of the Royal Statistical Society Series B"}],"container-title":["Foundations and Trends\u00ae in Machine Learning"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.emerald.com\/ftmal\/article-pdf\/4\/1\/1\/11135038\/2200000015en.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/www.emerald.com\/ftmal\/article-pdf\/4\/1\/1\/11135038\/2200000015en.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T18:10:41Z","timestamp":1777486241000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.emerald.com\/ftmal\/article\/4\/1\/1\/1331297\/Optimization-with-Sparsity-Inducing-Penalties"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,1,4]]},"references-count":159,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2012,1,4]]}},"URL":"https:\/\/doi.org\/10.1561\/2200000015","relation":{},"ISSN":["1935-8237","1935-8245"],"issn-type":[{"value":"1935-8237","type":"print"},{"value":"1935-8245","type":"electronic"}],"subject":[],"published":{"date-parts":[[2012,1,4]]}}}