{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,12]],"date-time":"2026-04-12T15:41:46Z","timestamp":1776008506884,"version":"3.50.1"},"reference-count":43,"publisher":"Oxford University Press (OUP)","license":[{"start":{"date-parts":[[2026,4,9]],"date-time":"2026-04-09T00:00:00Z","timestamp":1775692800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/100000006","name":"Office of Naval Research","doi-asserted-by":"publisher","award":["N000142512504"],"award-info":[{"award-number":["N000142512504"]}],"id":[{"id":"10.13039\/100000006","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>We study the high-dimensional linear regression problem with categorical predictors that have many levels. We propose a new estimation approach, which performs model compression via two mechanisms by simultaneously encouraging (a) clustering of the regression coefficients to collapse some of the categorical levels together; and (b) sparsity of the regression coefficients. We present novel mixed integer programming formulations for our estimator, and develop a custom row generation procedure to speed up the exact off-the-shelf solvers. We also propose a fast approximate algorithm for our method that obtains high-quality feasible solutions via block coordinate descent. As the main building block of our algorithm, we develop an exact algorithm for the univariate case based on dynamic programming, which can be of independent interest. We establish new theoretical guarantees for both the prediction and the cluster recovery performance of our estimator. Our numerical experiments on synthetic and real datasets demonstrate that our proposed estimator tends to outperform the state-of-the-art.<\/jats:p>","DOI":"10.1093\/jrsssb\/qkag062","type":"journal-article","created":{"date-parts":[[2026,3,28]],"date-time":"2026-03-28T12:47:54Z","timestamp":1774702074000},"source":"Crossref","is-referenced-by-count":0,"title":["Modelling with categorical features via exact fusion and sparsity regularization"],"prefix":"10.1093","author":[{"given":"Kayhan","family":"Behdin","sequence":"first","affiliation":[{"name":"Operations Research Center, MIT , Cambridge, MA 02139 ,","place":["USA"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Riade","family":"Benbaki","sequence":"additional","affiliation":[{"name":"Operations Research Center, MIT , Cambridge, MA 02139 ,","place":["USA"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peter","family":"Radchenko","sequence":"additional","affiliation":[{"name":"The University of Sydney Business School, University of Sydney , Sydney, NSW 2006 ,","place":["Australia"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rahul","family":"Mazumder","sequence":"additional","affiliation":[{"name":"Operations Research Center, MIT , Cambridge, MA 02139 ,","place":["USA"]},{"name":"Sloan School of Management, MIT , Cambridge, MA 02139 ,","place":["USA"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2026,4,9]]},"reference":[{"issue":"2","key":"2026041210461807200_qkag062-B1","doi-asserted-by":"publisher","first-page":"813","DOI":"10.1214\/15-AOS1388","article-title":"Best subset selection via a modern optimization lens","volume":"44","author":"Bertsimas","year":"2016","journal-title":"Annals of Statistics"},{"issue":"1","key":"2026041210461807200_qkag062-B2","doi-asserted-by":"publisher","first-page":"300","DOI":"10.1214\/18-AOS1804","article-title":"Sparse high-dimensional regression","volume":"48","author":"Bertsimas","year":"2020","journal-title":"Annals of Statistics"},{"issue":"3","key":"2026041210461807200_qkag062-B3","doi-asserted-by":"publisher","first-page":"265","DOI":"10.1016\/j.acha.2009.04.002","article-title":"Iterative hard thresholding for compressed sensing","volume":"27","author":"Blumensath","year":"2009","journal-title":"Applied and Computational Harmonic Analysis"},{"issue":"1","key":"2026041210461807200_qkag062-B4","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1111\/j.1541-0420.2007.00843.x","article-title":"Simultaneous regression shrinkage, variable selection, and supervised clustering of predictors with Oscar","volume":"64","author":"Bondell","year":"2008","journal-title":"Biometrics"},{"issue":"1","key":"2026041210461807200_qkag062-B5","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1111\/j.1541-0420.2008.01061.x","article-title":"Simultaneous factor selection and collapsing levels in ANOVA","volume":"65","author":"Bondell","year":"2009","journal-title":"Biometrics"},{"issue":"1","key":"2026041210461807200_qkag062-B6","first-page":"6008","article-title":"Learning sparse classifiers: Continuous and mixed integer optimization perspectives","volume":"22","author":"Dedieu","year":"2021","journal-title":"Journal of Machine Learning Research: JMLR"},{"issue":"456","key":"2026041210461807200_qkag062-B7","doi-asserted-by":"publisher","first-page":"1348","DOI":"10.1198\/016214501753382273","article-title":"Variable selection via nonconcave penalized likelihood and its oracle properties","volume":"96","author":"Fan","year":"2001","journal-title":"Journal of the American Statistical Association"},{"key":"2026041210461807200_qkag062-B8","doi-asserted-by":"publisher","first-page":"113","DOI":"10.1007\/s13748-013-0040-3","article-title":"Event labeling combining ensemble detectors and background knowledge","volume":"2","author":"Fanaee-T","year":"2014","journal-title":"Progress in Artificial Intelligence"},{"key":"2026041210461807200_qkag062-B9","doi-asserted-by":"publisher","first-page":"2150","DOI":"10.1214\/10-AOAS355","article-title":"Sparse modeling of categorical explanatory variables","volume":"4","author":"Gertheiss","year":"2010","journal-title":"The Annals of Applied Statistics"},{"issue":"5","key":"2026041210461807200_qkag062-B10","doi-asserted-by":"publisher","first-page":"2367","DOI":"10.1214\/009053606000000768","article-title":"Best subset selection, persistence in high-dimensional statistical learning and optimization under L1 constraint","volume":"34","author":"Greenshtein","year":"2006","journal-title":"Annals of Statistics"},{"key":"2026041210461807200_qkag062-B11","doi-asserted-by":"crossref","DOI":"10.1201\/b18401","volume-title":"Statistical learning with sparsity","author":"Hastie","year":"2015"},{"issue":"5","key":"2026041210461807200_qkag062-B12","doi-asserted-by":"publisher","first-page":"1517","DOI":"10.1287\/opre.2019.1919","article-title":"Fast best subset selection: Coordinate descent and local combinatorial optimization algorithms","volume":"68","author":"Hazimeh","year":"2020","journal-title":"Operations Research"},{"issue":"1","key":"2026041210461807200_qkag062-B13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1214\/21-AOS2155","article-title":"Grouped variable selection with discrete optimization: Computational and statistical perspectives","volume":"51","author":"Hazimeh","year":"2023","journal-title":"Annals of Statistics"},{"issue":"1-2","key":"2026041210461807200_qkag062-B14","doi-asserted-by":"publisher","first-page":"347","DOI":"10.1007\/s10107-021-01712-4","article-title":"Sparse regression at scale: Branch-and-bound rooted in first-order optimization","volume":"196","author":"Hazimeh","year":"2022","journal-title":"Mathematical Programming"},{"issue":"1","key":"2026041210461807200_qkag062-B15","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1080\/00401706.1970.10488634","article-title":"Ridge regression: Biased estimation for nonorthogonal problems","volume":"12","author":"Hoerl","year":"1970","journal-title":"Technometrics: A Journal of Statistics for the Physical, Chemical, and Engineering Sciences"},{"issue":"1","key":"2026041210461807200_qkag062-B16","doi-asserted-by":"publisher","first-page":"e180","DOI":"10.1002\/sta4.180","article-title":"Motor insurance claim modelling with factor collapsing and Bayesian model averaging","volume":"7","author":"Hu","year":"2018","journal-title":"Stat"},{"issue":"6","key":"2026041210461807200_qkag062-B17","doi-asserted-by":"publisher","first-page":"395","DOI":"10.1038\/nrg3208","article-title":"Mining electronic health records: Towards better research applications and clinical care","volume":"13","author":"Jensen","year":"2012","journal-title":"Nature Reviews: Genetics"},{"issue":"2","key":"2026041210461807200_qkag062-B18","doi-asserted-by":"publisher","first-page":"246","DOI":"10.1080\/10618600.2012.681238","article-title":"A dynamic programming algorithm for the fused lasso and l 0-segmentation","volume":"22","author":"Johnson","year":"2013","journal-title":"Journal of Computational and Graphical Statistics"},{"issue":"509","key":"2026041210461807200_qkag062-B19","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1080\/01621459.2014.892882","article-title":"Homogeneity pursuit","volume":"110","author":"Ke","year":"2015","journal-title":"Journal of the American Statistical Association"},{"key":"2026041210461807200_qkag062-B20","doi-asserted-by":"crossref","DOI":"10.1201\/9781315108230","volume-title":"Feature engineering and selection: A practical approach for predictive models","author":"Kuhn","year":"2019"},{"key":"2026041210461807200_qkag062-B21","doi-asserted-by":"publisher","author":"Lu","year":"2016","DOI":"10.48550\/arXiv.1612.02099"},{"key":"2026041210461807200_qkag062-B22","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511809071","volume-title":"Introduction to information retrieval","author":"Manning","year":"2008"},{"issue":"1","key":"2026041210461807200_qkag062-B23","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1287\/opre.2022.2276","article-title":"Subset selection with shrinkage: Sparse linear modeling when the SNR is low","volume":"71","author":"Mazumder","year":"2023","journal-title":"Operations Research"},{"key":"2026041210461807200_qkag062-B24","doi-asserted-by":"crossref","DOI":"10.1201\/9781420035933","volume-title":"Subset selection in regression","author":"Miller","year":"2002"},{"key":"2026041210461807200_qkag062-B25","author":"Nowakowski","year":"2023"},{"issue":"5","key":"2026041210461807200_qkag062-B26","doi-asserted-by":"publisher","first-page":"389","DOI":"10.1177\/1471082X14553366","article-title":"Selection and fusion of categorical predictors with l0-type penalties","volume":"15","author":"Oelker","year":"2015","journal-title":"Statistical Modelling"},{"issue":"5","key":"2026041210461807200_qkag062-B27","doi-asserted-by":"publisher","first-page":"1527","DOI":"10.1111\/rssb.12226","article-title":"Convex clustering via \u21131 fusion penalization","volume":"79","author":"Radchenko","year":"2017","journal-title":"Journal of the Royal Statistical Society: Series B, Statistical Methodology"},{"issue":"10","key":"2026041210461807200_qkag062-B28","doi-asserted-by":"publisher","first-page":"6976","DOI":"10.1109\/TIT.2011.2165799","article-title":"Minimax rates of estimation for high-dimensional linear regression over \u2113q -balls","volume":"57","author":"Raskutti","year":"2011","journal-title":"IEEE Transactions on Information Theory"},{"key":"2026041210461807200_qkag062-B29","author":"Reuther","year":"2018"},{"key":"2026041210461807200_qkag062-B30","doi-asserted-by":"publisher","first-page":"1055","DOI":"10.1214\/10-EJS578","article-title":"Sparse regression with exact clustering","volume":"4","author":"She","year":"2010","journal-title":"Electronic Journal of Statistics"},{"issue":"3","key":"2026041210461807200_qkag062-B31","doi-asserted-by":"publisher","first-page":"912","DOI":"10.1111\/rssb.12492","article-title":"Supervised multivariate learning with simultaneous feature auto-grouping and dimension reduction","volume":"84","author":"She","year":"2022","journal-title":"Journal of the Royal Statistical Society: Series B, Statistical Methodology"},{"issue":"490","key":"2026041210461807200_qkag062-B32","doi-asserted-by":"publisher","first-page":"727","DOI":"10.1198\/jasa.2010.tm09380","article-title":"Grouping pursuit through a regularization solution surface","volume":"105","author":"Shen","year":"2010","journal-title":"Journal of the American Statistical Association"},{"issue":"3","key":"2026041210461807200_qkag062-B33","doi-asserted-by":"publisher","first-page":"579","DOI":"10.1111\/rssb.12432","article-title":"Modelling high-dimensional categorical data using nonconvex fusion penalties","volume":"83","author":"Stokell","year":"2021","journal-title":"Journal of the Royal Statistical Society: Series B, Statistical Methodology"},{"issue":"1","key":"2026041210461807200_qkag062-B34","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression shrinkage and selection via the lasso","volume":"58","author":"Tibshirani","year":"1996","journal-title":"Journal of the Royal Statistical Society. Series B (Methodological)"},{"issue":"1","key":"2026041210461807200_qkag062-B35","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1111\/j.1467-9868.2005.00490.x","article-title":"Sparsity and smoothness via the fused lasso","volume":"67","author":"Tibshirani","year":"2005","journal-title":"Journal of the Royal Statistical Society: Series B, Statistical Methodology"},{"issue":"3","key":"2026041210461807200_qkag062-B36","doi-asserted-by":"publisher","first-page":"475","DOI":"10.1023\/A:1017501703105","article-title":"Convergence of a block coordinate descent method for nondifferentiable minimization","volume":"109","author":"Tseng","year":"2001","journal-title":"Journal of Optimization Theory and Applications"},{"issue":"3","key":"2026041210461807200_qkag062-B37","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1177\/1471082X16642560","article-title":"Regularized regression for categorical data","volume":"16","author":"Tutz","year":"2016","journal-title":"Statistical Modelling"},{"issue":"1","key":"2026041210461807200_qkag062-B38","doi-asserted-by":"publisher","first-page":"285","DOI":"10.1214\/23-aos2347","article-title":"Supervised homogeneity fusion: A combinatorial approach","volume":"52","author":"Wang","year":"2024","journal-title":"Annals of Statistics"},{"key":"2026041210461807200_qkag062-B39","volume-title":"Integer and combinatorial optimization","author":"Wolsey","year":"1999"},{"issue":"1","key":"2026041210461807200_qkag062-B40","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1111\/j.1467-9868.2005.00532.x","article-title":"Model selection and estimation in regression with grouped variables","volume":"68","author":"Yuan","year":"2006","journal-title":"Journal of the Royal Statistical Society: Series B, Statistical Methodology"},{"issue":"2","key":"2026041210461807200_qkag062-B41","doi-asserted-by":"publisher","first-page":"894","DOI":"10.1214\/09-AOS729","article-title":"Nearly unbiased variable selection under minimax concave penalty","volume":"38","author":"Zhang","year":"2010","journal-title":"Annals of Statistics"},{"issue":"502","key":"2026041210461807200_qkag062-B42","doi-asserted-by":"publisher","first-page":"713","DOI":"10.1080\/01621459.2013.770704","article-title":"Simultaneous grouping pursuit and feature selection over an undirected graph","volume":"108","author":"Zhu","year":"2013","journal-title":"Journal of the American Statistical Association"},{"issue":"2","key":"2026041210461807200_qkag062-B43","doi-asserted-by":"publisher","first-page":"301","DOI":"10.1111\/j.1467-9868.2005.00503.x","article-title":"Regularization and variable selection via the elastic net","volume":"67","author":"Zou","year":"2005","journal-title":"Journal of the Royal Statistical Society: Series B, Statistical Methodology"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jrsssb\/advance-article-pdf\/doi\/10.1093\/jrsssb\/qkag062\/68018036\/qkag062.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/advance-article-pdf\/doi\/10.1093\/jrsssb\/qkag062\/68018036\/qkag062.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,12]],"date-time":"2026-04-12T14:46:22Z","timestamp":1776005182000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/advance-article\/doi\/10.1093\/jrsssb\/qkag062\/8650382"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,9]]},"references-count":43,"URL":"https:\/\/doi.org\/10.1093\/jrsssb\/qkag062","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,9]]},"article-number":"qkag062"}}