{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T12:55:09Z","timestamp":1790686509540,"version":"4.1.0"},"reference-count":44,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2023,2,24]],"date-time":"2023-02-24T00:00:00Z","timestamp":1677196800000},"content-version":"vor","delay-in-days":603,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,7,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We propose a method for estimation in high-dimensional linear models with nominal categorical data. Our estimator, called SCOPE, fuses levels together by making their corresponding coefficients exactly equal. This is achieved using the minimax concave penalty on differences between the order statistics of the coefficients for a categorical variable, thereby clustering the coefficients. We provide an algorithm for exact and efficient computation of the global minimum of the resulting nonconvex objective in the case with a single variable with potentially many levels, and use this within a block coordinate descent procedure in the multivariate case. We show that an oracle least squares solution that exploits the unknown level fusions is a limit point of the coordinate descent with high probability, provided the true levels have a certain minimum separation; these conditions are known to be minimal in the univariate case. We demonstrate the favourable performance of SCOPE across a range of real and simulated datasets. An R package CatReg implementing SCOPE for linear models and also a version for logistic regression is available on CRAN.<\/jats:p>","DOI":"10.1111\/rssb.12432","type":"journal-article","created":{"date-parts":[[2021,7,22]],"date-time":"2021-07-22T12:40:36Z","timestamp":1626957636000},"page":"579-611","source":"Crossref","is-referenced-by-count":19,"title":["Modelling High-Dimensional Categorical Data using Nonconvex Fusion Penalties"],"prefix":"10.1093","volume":"83","author":[{"given":"Benjamin G.","family":"Stokell","sequence":"first","affiliation":[{"name":"University of Cambridge , Cambridge , UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rajen D.","family":"Shah","sequence":"additional","affiliation":[{"name":"University of Cambridge , Cambridge , UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ryan J.","family":"Tibshirani","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University , Pittsburgh, PA , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2021,7,22]]},"reference":[{"issue":"1","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1111\/j.1541-0420.2008.01061.x","article-title":"Simultaneous factor selection and collapsing levels in ANOVA","volume":"65","author":"Bondell","year":"2009","journal-title":"Biometrics"},{"issue":"1","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1214\/10-AOAS388","article-title":"Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection","volume":"5","author":"Breheny","year":"2011","journal-title":"The Annals of Applied Statistics"},{"issue":"2","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1007\/s11222-013-9424-2","article-title":"Group descent algorithms for nonconvex penalized linear and logistic regression models with grouped predictors","volume":"25","author":"Breheny","year":"2015","journal-title":"Statistics and Computing"},{"issue":"1","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Machine Learning"},{"key":"2023022408371788200_","article-title":"The Wadsworth and Brooks-Cole statistics-probability series.","volume-title":"Classification and regression trees","author":"Breiman","year":"1984"},{"key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"39","DOI":"10.2307\/2530641","article-title":"Clustering means in ANOVA by simultaneous testing","volume":"41","author":"Calinski","year":"1985","journal-title":"Biometrics"},{"issue":"3","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"759","DOI":"10.1093\/biomet\/asn034","article-title":"Extended Bayesian information criteria for model selection with large model spaces","volume":"95","author":"Chen","year":"2008","journal-title":"Biometrika"},{"issue":"1","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"205","DOI":"10.1080\/10618600.2015.1096789","article-title":"Fast tree inference with weighted fusion penalties","volume":"26","author":"Chiquet","year":"2017","journal-title":"Journal of Computational and Graphical Statistics"},{"key":"2023022408371788200_","volume-title":"Forecast Hub","author":"COVID-19 Forecast Hub","year":"2020"},{"key":"2023022408371788200_","volume-title":"UCI machine learning repository","author":"Dua","year":"2019"},{"issue":"456","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"1348","DOI":"10.1198\/016214501753382273","article-title":"Variable selection via nonconcave penalized likelihood and its oracle properties","volume":"96","author":"Fan","year":"2001","journal-title":"Journal of the American Statistical Association"},{"issue":"2","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"814","DOI":"10.1214\/17-AOS1568","article-title":"I-LAMM for sparse learning: Simultaneous control of algorithmic complexity and statistical error","volume":"46","author":"Fan","year":"2018","journal-title":"The Annals of Statistics"},{"issue":"1","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v033.i01","article-title":"Regularization paths for generalized linear models via coordinate descent","volume":"33","author":"Friedman","year":"2010","journal-title":"Journal of Statistical Software"},{"issue":"4","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"2150","DOI":"10.1214\/10-AOAS355","article-title":"Sparse modeling of categorial explanatory variables","volume":"4","author":"Gertheiss","year":"2010","journal-title":"The Annals of Applied Statistics"},{"key":"2023022408371788200_","first-page":"1","article-title":"Clusterpath an algorithm for clustering using convex fusion penalties","volume-title":"28th International Conference on Machine Learning","author":"Hocking","year":"2011"},{"issue":"1","key":"2023022408371788200_","doi-asserted-by":"crossref","DOI":"10.1002\/sta4.180","article-title":"Motor insurance claim modelling with factor collapsing and bayesian model averaging","volume":"7","author":"Hu","year":"2018","journal-title":"Stat"},{"issue":"1","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1007\/BF01908075","article-title":"Comparing partitions","volume":"2","author":"Hubert","year":"1985","journal-title":"Journal of Classification"},{"issue":"6","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"395","DOI":"10.1038\/nrg3208","article-title":"Mining electronic health records: towards better research applications and clinical care","volume":"13","author":"Jensen","year":"2012","journal-title":"Nature Reviews Genetics"},{"key":"2023022408371788200_","volume-title":"Prudential life insurance assessment","author":"Kaggle","year":"2015"},{"issue":"3","key":"2023022408371788200_","first-page":"18","article-title":"Classification and regression by randomforest","volume":"2","author":"Liaw","year":"2002","journal-title":"R News"},{"issue":"3","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"1637","DOI":"10.1214\/12-AOS1018","article-title":"High-dimensional regression with noisy and missing data: provable guarantees with nonconvexity","volume":"40","author":"Loh","year":"2012","journal-title":"The Annals of Statistics"},{"issue":"19","key":"2023022408371788200_","first-page":"559","article-title":"Regularized M-estimators with nonconvexity: Statistical and algorithmic theory for local optima","volume":"16","author":"Loh","year":"2015","journal-title":"Journal of Machine Learning Research"},{"key":"2023022408371788200_","volume-title":"Statistical and computational guarantees of lloyd's algorithm and its variants","author":"Lu","year":"2016"},{"issue":"517","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"410","DOI":"10.1080\/01621459.2016.1148039","article-title":"A concave pairwise fusion approach to subgroup analysis","volume":"112","author":"Ma","year":"2017","journal-title":"Journal of the American Statistical Association"},{"issue":"2","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"1749","DOI":"10.1214\/15-EJS1050","article-title":"Delete or merge regressors for linear model selection","volume":"9","author":"Maj-Ka\u0144ska","year":"2015","journal-title":"Electronic Journal of Statistics"},{"issue":"495","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"1125","DOI":"10.1198\/jasa.2011.tm09738","article-title":"Sparsenet: coordinate descent with nonconvex penalties","volume":"106","author":"Mazumder","year":"2011","journal-title":"Journal of the American Statistical Association"},{"issue":"5","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1177\/1471082X14553366","article-title":"Selection and fusion of categorical predictors with l 0-type penalties","volume":"15","author":"Oelker","year":"2015","journal-title":"Statistical Modelling"},{"issue":"2","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1214\/18-BA1096","article-title":"Bayesian effect fusion for categorical predictors","volume":"14","author":"Pauger","year":"2019","journal-title":"Bayesian Analysis"},{"key":"2023022408371788200_","volume-title":"EffectFusion: Bayesian effect fusion for categorical predictors","author":"Pauger","year":"2019"},{"key":"2023022408371788200_","volume-title":"DMRnet: delete or merge regressors algorithms for linear and logistic model selection and high-dimensional data","author":"Prochenka-Sotys","year":"2018"},{"key":"2023022408371788200_","volume-title":"R: A language and environment for statistical computing","author":"R Core Team","year":"2020"},{"key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"507","DOI":"10.2307\/2529204","article-title":"A cluster analysis method for grouping means in the analysis of variance","volume":"30","author":"Scott","year":"1974","journal-title":"Biometrics"},{"key":"2023022408371788200_","volume-title":"CatReg: solution paths for linear and logistic regression models with categorical predictors, with SCOPE penalty","author":"Stokell","year":"2021"},{"key":"2023022408371788200_","article-title":"rpart: recursive partitioning and regression trees","author":"Therneau","year":"2019"},{"issue":"1","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression shrinkage and selection via the lasso","volume":"58","author":"Tibshirani","year":"1996","journal-title":"Journal of the Royal Statistical Society: Series B (Methodological)"},{"issue":"1","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1111\/j.1467-9868.2005.00490.x","article-title":"Sparsity and smoothness via the fused lasso","volume":"67","author":"Tibshirani","year":"2005","journal-title":"Journal of the Royal Statistical Society: Series B (Statistical Methodology)"},{"issue":"3","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"475","DOI":"10.1023\/A:1017501703105","article-title":"Convergence of a block coordinate descent method for nondifferentiable minimization","volume":"109","author":"Tseng","year":"2001","journal-title":"Journal of Optimization Theory and Applications"},{"key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"99","DOI":"10.2307\/3001913","article-title":"Comparing individual means in the analysis of variance","volume":"5","author":"Tukey","year":"1949","journal-title":"Biometrics"},{"issue":"3","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1177\/1471082X16642560","article-title":"Regularized regression for categorical data","volume":"16","author":"Tutz","year":"2016","journal-title":"Statistical Modelling"},{"issue":"6","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"2164","DOI":"10.1214\/14-AOS1238","article-title":"Optimal computational and statistical rates of convergence for sparse nonconvex learning problems","volume":"42","author":"Wang","year":"2014","journal-title":"The Annals of Statistics"},{"issue":"1","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1111\/j.1467-9868.2005.00532.x","article-title":"Model selection and estimation in regression with grouped variables","volume":"68","author":"Yuan","year":"2006","journal-title":"Journal of the Royal Statistical Society: Series B (Statistical Methodology)"},{"issue":"2","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"894","DOI":"10.1214\/09-AOS729","article-title":"Nearly unbiased variable selection under minimax concave penalty","volume":"38","author":"Zhang","year":"2010","journal-title":"The Annals of Statistics"},{"issue":"1","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1214\/17-AOS1547","article-title":"Pathwise coordinate optimization for sparse learning: algorithm and theory","volume":"46","author":"Zhao","year":"2018","journal-title":"The Annals of Statistics"},{"issue":"476","key":"2023022408371788200_","doi-asserted-by":"crossref","first-page":"1418","DOI":"10.1198\/016214506000000735","article-title":"The adaptive lasso and its oracle properties","volume":"101","author":"Zou","year":"2006","journal-title":"Journal of the American Statistical Association"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/rssb.12432","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1111\/rssb.12432","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/83\/3\/579\/49321485\/jrsssb_83_3_579.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/83\/3\/579\/49321485\/jrsssb_83_3_579.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,4]],"date-time":"2024-09-04T21:01:56Z","timestamp":1725483716000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/article\/83\/3\/579\/7056093"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,1]]},"references-count":44,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2021,7,22]]},"published-print":{"date-parts":[[2021,7,1]]}},"URL":"https:\/\/doi.org\/10.1111\/rssb.12432","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,7]]},"published":{"date-parts":[[2021,7,1]]}}}