{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,7]],"date-time":"2026-04-07T07:17:02Z","timestamp":1775546222326,"version":"3.50.1"},"reference-count":30,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T00:00:00Z","timestamp":1755734400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T00:00:00Z","timestamp":1755734400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100018693","name":"HORIZON EUROPE Framework Programme","doi-asserted-by":"publisher","award":["101084642"],"award-info":[{"award-number":["101084642"]}],"id":[{"id":"10.13039\/100018693","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Comput Optim Appl"],"published-print":{"date-parts":[[2026,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    For statistical modeling wherein the data regime is unfavorable in terms of dimensionality relative to the sample size, finding hidden sparsity in the relationship structure between variables can be critical in formulating an accurate statistical model. The so-called \u201c\n                    <jats:inline-formula>\n                      <jats:tex-math>$$\\ell _0$$<\/jats:tex-math>\n                    <\/jats:inline-formula>\n                    norm\u201d, which counts the number of non-zero components in a vector, is a strong reliable mechanism of enforcing sparsity when incorporated into an optimization problem for minimizing the fit of a given model to a set of observations. However, in big data settings wherein noisy estimates of the gradient must be evaluated out of computational necessity, the literature is scant on methods that reliably converge. In this paper, we present an approach towards solving expectation objective optimization problems with cardinality constraints. We prove convergence of the underlying stochastic process and demonstrate the performance on two Machine Learning problems.\n                  <\/jats:p>","DOI":"10.1007\/s10589-025-00724-6","type":"journal-article","created":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T21:43:52Z","timestamp":1755812632000},"page":"57-83","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Probabilistic iterative hard thresholding for sparse learning"],"prefix":"10.1007","volume":"93","author":[{"given":"Matteo","family":"Bergamaschi","sequence":"first","affiliation":[]},{"given":"Andrea","family":"Cristofari","sequence":"additional","affiliation":[]},{"given":"Vyacheslav","family":"Kungurtsev","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8978-6027","authenticated-orcid":false,"given":"Francesco","family":"Rinaldi","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,8,21]]},"reference":[{"issue":"2","key":"724_CR1","doi-asserted-by":"publisher","first-page":"219","DOI":"10.1007\/s10898-021-01070-7","volume":"82","author":"S L\u00e4mmel","year":"2022","unstructured":"L\u00e4mmel, S., Shikhman, V.: On nondegenerate m-stationary points for sparsity constrained nonlinear optimization. J. Global Optim. 82(2), 219\u2013242 (2022)","journal-title":"J. Global Optim."},{"issue":"2","key":"724_CR2","doi-asserted-by":"publisher","first-page":"521","DOI":"10.1080\/02331934.2021.1981317","volume":"72","author":"S L\u00e4mmel","year":"2023","unstructured":"L\u00e4mmel, S., Shikhman, V.: Critical point theory for sparse recovery. Optimization 72(2), 521\u2013549 (2023)","journal-title":"Optimization"},{"issue":"3","key":"724_CR3","doi-asserted-by":"publisher","first-page":"1480","DOI":"10.1137\/120869778","volume":"23","author":"A Beck","year":"2013","unstructured":"Beck, A., Eldar, Y.C.: Sparsity constrained nonlinear optimization: optimality conditions and algorithms. SIAM J. Optim. 23(3), 1480\u20131509 (2013)","journal-title":"SIAM J. Optim."},{"issue":"1","key":"724_CR4","doi-asserted-by":"publisher","first-page":"196","DOI":"10.1287\/moor.2015.0722","volume":"41","author":"A Beck","year":"2016","unstructured":"Beck, A., Hallak, N.: On the minimization over sparse symmetric sets: projections, optimality conditions, and algorithms. Math. Oper. Res. 41(1), 196\u2013223 (2016)","journal-title":"Math. Oper. Res."},{"issue":"1","key":"724_CR5","doi-asserted-by":"publisher","first-page":"790","DOI":"10.1137\/22M1535498","volume":"34","author":"N Hallak","year":"2024","unstructured":"Hallak, N.: A path-based approach to constrained sparse optimization. SIAM J. Optim. 34(1), 790\u2013816 (2024)","journal-title":"SIAM J. Optim."},{"issue":"2","key":"724_CR6","doi-asserted-by":"publisher","first-page":"503","DOI":"10.1007\/s10589-018-9985-2","volume":"70","author":"M Branda","year":"2018","unstructured":"Branda, M., Bucher, M., \u010cervinka, M., Schwartz, A.: Convergence of a Scholtes-type regularization method for cardinality-constrained optimization problems with an application in sparse robust portfolio optimization. Comput. Optim. Appl. 70(2), 503\u2013530 (2018)","journal-title":"Comput. Optim. Appl."},{"issue":"2","key":"724_CR7","doi-asserted-by":"publisher","first-page":"383","DOI":"10.1007\/s10957-018-1320-7","volume":"178","author":"M Bucher","year":"2018","unstructured":"Bucher, M., Schwartz, A.: Second-order optimality conditions and improved convergence results for regularization methods for cardinality-constrained optimization problems. J. Optim. Theory Appl. 178(2), 383\u2013410 (2018)","journal-title":"J. Optim. Theory Appl."},{"issue":"1","key":"724_CR8","doi-asserted-by":"publisher","first-page":"397","DOI":"10.1137\/140978077","volume":"26","author":"O Burdakov","year":"2016","unstructured":"Burdakov, O., Kanzow, C., Schwartz, A.: Mathematical programs with cardinality constraints: reformulation by complementarity-type conditions and a regularization method. SIAM J. Optim. 26(1), 397\u2013425 (2016)","journal-title":"SIAM J. Optim."},{"issue":"1","key":"724_CR9","doi-asserted-by":"publisher","first-page":"353","DOI":"10.1007\/s10107-016-0986-6","volume":"160","author":"M \u010cervinka","year":"2016","unstructured":"\u010cervinka, M., Kanzow, C., Schwartz, A.: Constraint qualifications and optimality conditions for optimization problems with cardinality constraints. Math. Program. 160(1), 353\u2013377 (2016)","journal-title":"Math. Program."},{"issue":"2","key":"724_CR10","doi-asserted-by":"publisher","first-page":"473","DOI":"10.1007\/s10957-020-01793-9","volume":"188","author":"M Lapucci","year":"2021","unstructured":"Lapucci, M., Levato, T., Sciandrone, M.: Convergent inexact penalty decomposition methods for cardinality-constrained problems. J. Optim. Theory Appl. 188(2), 473\u2013496 (2021)","journal-title":"J. Optim. Theory Appl."},{"issue":"2","key":"724_CR11","doi-asserted-by":"publisher","first-page":"663","DOI":"10.1007\/s10957-023-02306-0","volume":"199","author":"M Lapucci","year":"2023","unstructured":"Lapucci, M., Levato, T., Rinaldi, F., Sciandrone, M.: A unifying framework for sparsity-constrained optimization. J. Optim. Theory Appl. 199(2), 663\u2013692 (2023)","journal-title":"J. Optim. Theory Appl."},{"issue":"4","key":"724_CR12","doi-asserted-by":"publisher","first-page":"2448","DOI":"10.1137\/100808071","volume":"23","author":"Z Lu","year":"2013","unstructured":"Lu, Z., Zhang, Y.: Sparse approximation via penalty decomposition methods. SIAM J. Optim. 23(4), 2448\u20132478 (2013)","journal-title":"SIAM J. Optim."},{"key":"724_CR13","unstructured":"Zhou, P., Yuan, X., Feng, J.: Efficient stochastic gradient hard thresholding. Adv. Neural Inform. Process. Syst. 31 (2018)"},{"key":"724_CR14","unstructured":"Zhou, B., Chen, F., Ying, Y.: Stochastic iterative hard thresholding for graph-structured sparsity optimization. In: International Conference on Machine Learning, pp. 7563\u20137573 (2019). PMLR"},{"key":"724_CR15","unstructured":"Murata, T., Suzuki, T.: Sample efficient stochastic gradient iterative hard thresholding method for stochastic sparse linear regression with limited attribute observation. Adv. Neural Inform. Process. Syst. 31 (2018)"},{"key":"724_CR16","unstructured":"Jain, P., Tewari, A., Kar, P.: On iterative hard thresholding methods for high-dimensional m-estimation. Adv. Neural Inform. Process. Syst. 27 (2014)"},{"key":"724_CR17","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1007\/s10287-005-0044-y","volume":"3","author":"F Bastin","year":"2006","unstructured":"Bastin, F., Cirillo, C., Toint, P.L.: An adaptive monte Carlo algorithm for computing mixed logit estimators. CMS 3, 55\u201379 (2006)","journal-title":"CMS"},{"issue":"3","key":"724_CR18","doi-asserted-by":"publisher","first-page":"1238","DOI":"10.1137\/130915984","volume":"24","author":"AS Bandeira","year":"2014","unstructured":"Bandeira, A.S., Scheinberg, K., Vicente, L.N.: Convergence of trust-region methods based on probabilistic models. SIAM J. Optim. 24(3), 1238\u20131264 (2014)","journal-title":"SIAM J. Optim."},{"key":"724_CR19","doi-asserted-by":"publisher","first-page":"447","DOI":"10.1007\/s10107-017-1141-8","volume":"169","author":"R Chen","year":"2018","unstructured":"Chen, R., Menickelly, M., Scheinberg, K.: Stochastic optimization using a trust-region method and random models. Math. Program. 169, 447\u2013487 (2018)","journal-title":"Math. Program."},{"key":"724_CR20","doi-asserted-by":"crossref","unstructured":"Carlini, N., Wagner, D.: Towards evaluating the robustness of neural networks. In: 2017 IEEE Symposium on Security and Privacy (SP), pp. 39\u201357 (2017). IEEE","DOI":"10.1109\/SP.2017.49"},{"key":"724_CR21","doi-asserted-by":"crossref","unstructured":"Croce, F., Hein, M.: Sparse and imperceivable adversarial attacks. In: Proceedings of the IEEE\/CVF International Conference on Computer Vision, pp. 4724\u20134732 (2019)","DOI":"10.1109\/ICCV.2019.00482"},{"key":"724_CR22","doi-asserted-by":"crossref","unstructured":"Modas, A., Moosavi-Dezfooli, S.-M., Frossard, P.: Sparsefool: a few pixels make a big difference. In: Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, pp. 9087\u20139096 (2019)","DOI":"10.1109\/CVPR.2019.00930"},{"key":"724_CR23","unstructured":"Behdin, K., Chen, W., Mazumder, R.: Sparse gaussian graphical models with discrete optimization: computational and statistical perspectives. Preprint at arXiv:2307.09366 (2023)"},{"key":"724_CR24","first-page":"25095","volume":"36","author":"MM Negri","year":"2023","unstructured":"Negri, M.M., Arend Torres, F., Roth, V.: Conditional matrix flows for gaussian graphical models. Adv. Neural. Inf. Process. Syst. 36, 25095\u201325111 (2023)","journal-title":"Adv. Neural. Inf. Process. Syst."},{"key":"724_CR25","doi-asserted-by":"publisher","first-page":"185","DOI":"10.1007\/s10589-021-00298-z","volume":"80","author":"C Kanzow","year":"2021","unstructured":"Kanzow, C., Raharja, A.B., Schwartz, A.: Sequential optimality conditions for cardinality-constrained optimization problems with applications. Comput. Optim. Appl. 80, 185\u2013211 (2021)","journal-title":"Comput. Optim. Appl."},{"key":"724_CR26","doi-asserted-by":"publisher","first-page":"629","DOI":"10.1007\/s00041-008-9035-z","volume":"14","author":"T Blumensath","year":"2008","unstructured":"Blumensath, T., Davies, M.E.: Iterative thresholding for sparse approximations. J. Fourier Anal. Appl. 14, 629\u2013654 (2008)","journal-title":"J. Fourier Anal. Appl."},{"key":"724_CR27","volume-title":"Nonlinear Programming","author":"DP Bertsekas","year":"1999","unstructured":"Bertsekas, D.P.: Nonlinear Programming. Athena Scientific, Belmont (1999)"},{"key":"724_CR28","doi-asserted-by":"publisher","first-page":"337","DOI":"10.1007\/s10107-017-1137-4","volume":"169","author":"C Cartis","year":"2018","unstructured":"Cartis, C., Scheinberg, K.: Global convergence rate analysis of unconstrained optimization methods based on probabilistic models. Math. Program. 169, 337\u2013375 (2018)","journal-title":"Math. Program."},{"key":"724_CR29","doi-asserted-by":"crossref","unstructured":"Carlini, N., Wagner, D.A.: Towards evaluating the robustness of neural networks. Preprint at arXiv:1608.04644 (2016)","DOI":"10.1109\/SP.2017.49"},{"key":"724_CR30","doi-asserted-by":"publisher","DOI":"10.1017\/9781108627771","volume-title":"High-Dimensional Statistics: A Non-Asymptotic Viewpoint","author":"MJ Wainwright","year":"2019","unstructured":"Wainwright, M.J.: High-Dimensional Statistics: A Non-Asymptotic Viewpoint. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press, Cambridge (2019)"}],"container-title":["Computational Optimization and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10589-025-00724-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10589-025-00724-6","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10589-025-00724-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,7]],"date-time":"2026-04-07T06:28:55Z","timestamp":1775543335000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10589-025-00724-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,21]]},"references-count":30,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,1]]}},"alternative-id":["724"],"URL":"https:\/\/doi.org\/10.1007\/s10589-025-00724-6","relation":{},"ISSN":["0926-6003","1573-2894"],"issn-type":[{"value":"0926-6003","type":"print"},{"value":"1573-2894","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,21]]},"assertion":[{"value":"24 October 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 August 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 August 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}