{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T01:39:07Z","timestamp":1790213947543,"version":"4.1.0"},"reference-count":70,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2021,12,14]],"date-time":"2021-12-14T00:00:00Z","timestamp":1639440000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,4,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Recent development in data-driven decision science has seen great advances in individualized decision making. Given data with individual covariates, treatment assignments and outcomes, researchers can search for the optimal individualized treatment rule (ITR) that maximizes the expected outcome. Existing methods typically require initial estimation of some nuisance models. The double robustness property that can protect from misspecification of either the treatment-free effect or the propensity score has been widely advocated. However, when model misspecification exists, a doubly robust estimate can be consistent but may suffer from downgraded efficiency. Other than potential misspecified nuisance models, most existing methods do not account for the potential problem when the variance of outcome is heterogeneous among covariates and treatment. We observe that such heteroscedasticity can greatly affect the estimation efficiency of the optimal ITR. In this paper, we demonstrate that the consequences of misspecified treatment-free effect and heteroscedasticity can be unified as a covariate-treatment dependent variance of residuals. To improve efficiency of the estimated ITR, we propose an Efficient Learning (E-Learning) framework for finding an optimal ITR in the multi-armed treatment setting. We show that the proposed E-Learning is optimal among a regular class of semiparametric estimates that can allow treatment-free effect misspecification. In our simulation study, E-Learning demonstrates its effectiveness if one of or both misspecified treatment-free effect and heteroscedasticity exist. Our analysis of a type 2 diabetes mellitus (T2DM) observational study also suggests the improved efficiency of E-Learning.<\/jats:p>","DOI":"10.1111\/rssb.12474","type":"journal-article","created":{"date-parts":[[2021,12,15]],"date-time":"2021-12-15T06:56:48Z","timestamp":1639551408000},"page":"440-472","source":"Crossref","is-referenced-by-count":9,"title":["Efficient Learning of Optimal Individualized Treatment Rules for Heteroscedastic or Misspecified Treatment-Free Effect Models"],"prefix":"10.1093","volume":"84","author":[{"given":"Weibin","family":"Mo","sequence":"first","affiliation":[{"name":"Department of Statistics and Operations Research , , Chapel Hill, North Carolina , USA"},{"name":"University of North Carolina at Chapel Hill , , Chapel Hill, North Carolina , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yufeng","family":"Liu","sequence":"additional","affiliation":[{"name":"Department of Statistics and Operations Research , , , , , , Chapel Hill, North Carolina , USA"},{"name":"Department of Genetics , , , , , , Chapel Hill, North Carolina , USA"},{"name":"Department of Biostatistics , , , , , , Chapel Hill, North Carolina , USA"},{"name":"Carolina Center for Genome Sciences , , , , , , Chapel Hill, North Carolina , USA"},{"name":"Lineberger Comprehensive Cancer Center , , , , , , Chapel Hill, North Carolina , USA"},{"name":"University of North Carolina at Chapel Hill , , , , , , Chapel Hill, North Carolina , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2021,12,14]]},"reference":[{"key":"2023022408360763100_","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-29659-3","volume-title":"Recommender systems: the textbook","author":"Aggarwal","year":"2016"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"284","DOI":"10.1080\/10618600.2012.681213","article-title":"Automatic feature selection via weighted kernels and regularization","volume":"22","author":"Allen","year":"2013","journal-title":"Journal of Computational and Graphical Statistics"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"133","DOI":"10.3982\/ECTA15732","article-title":"Policy learning with observational data","volume":"89","author":"Athey","year":"2021","journal-title":"Econometrica"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"1148","DOI":"10.1214\/18-AOS1709","article-title":"Generalized random forests","volume":"47","author":"Athey","year":"2019","journal-title":"The Annals of Statistics"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"863","DOI":"10.1093\/biomet\/asx053","article-title":"Doubly robust non-parametric inference on the average treatment effect","volume":"104","author":"Benkeser","year":"2017","journal-title":"Biometrika"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"210","DOI":"10.2337\/dc16-0826","article-title":"Personalized diabetes management using electronic medical records","volume":"40","author":"Bertsimas","year":"2017","journal-title":"Diabetes Care"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"723","DOI":"10.1093\/biomet\/asp033","article-title":"Improving efficiency and robustness of the doubly robust estimator for a population mean with incomplete data","volume":"96","author":"Cao","year":"2009","journal-title":"Biometrika"},{"key":"2023022408360763100_","first-page":"1224","article-title":"Adapting for heteroscedasticity in linear models","volume":"10","author":"Carroll","year":"1982","journal-title":"The Annals of Statistics"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"1199","DOI":"10.1111\/biom.12676","article-title":"A general statistical framework for subgroup identification and comparative treatment scoring","volume":"73","author":"Chen","year":"2017","journal-title":"Biometrics"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"924","DOI":"10.1111\/biom.12865","article-title":"Estimating individualized treatment rules for ordinal treatments","volume":"74","author":"Chen","year":"2018","journal-title":"Biometrics"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","DOI":"10.3386\/t0330","article-title":"Moving the goalposts: addressing limited overlap in the estimation of average treatment effects by changing the estimand","author":"Crump","year":"2006"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"1079","DOI":"10.1080\/01621459.1987.10478543","article-title":"Variance function estimation","volume":"82","author":"Davidian","year":"1987","journal-title":"Journal of the American Statistical Association"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"214","DOI":"10.1214\/18-STS645","article-title":"Causal inference: a missing data perspective","volume":"33","author":"Ding","year":"2018","journal-title":"Statistical Science"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"368","DOI":"10.1080\/01621459.2020.1753522","article-title":"Robust Q-learning","volume":"116","author":"Ertefaie","year":"2021","journal-title":"Journal of the American Statistical Association"},{"key":"2023022408360763100_","first-page":"1","article-title":"Multivariate adaptive regression splines","volume":"19","author":"Friedman","year":"1991","journal-title":"The Annals of Statistics"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v033.i01","article-title":"Regularization paths for generalized linear models via coordinate descent","volume":"33","author":"Friedman","year":"2010","journal-title":"Journal of Statistical Software"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"827","DOI":"10.1093\/ije\/dyv098","article-title":"Data resource profile: clinical practice research datalink (CPRD)","volume":"44","author":"Herrett","year":"2015","journal-title":"International Journal of Epidemiology"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"672","DOI":"10.1198\/016214508000000184","article-title":"Penalized estimating functions and variable selection in semiparametric regression models","volume":"103","author":"Johnson","year":"2008","journal-title":"Journal of the American Statistical Association"},{"key":"2023022408360763100_","first-page":"523","article-title":"Demystifying double robustness: a comparison of alternative strategies for estimating a population mean from incomplete data","volume":"22","author":"Kang","year":"2007","journal-title":"Statistical Science"},{"key":"2023022408360763100_","article-title":"Optimal doubly robust estimation of heterogeneous causal effects","volume-title":"arXiv preprint arXiv:2004.14497","author":"Kennedy","year":"2020"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"591","DOI":"10.3982\/ECTA13288","article-title":"Who should be treated? Empirical welfare maximization methods for treatment choice","volume":"86","author":"Kitagawa","year":"2018","journal-title":"Econometrica"},{"key":"2023022408360763100_","first-page":"622","article-title":"Allocating interventions based on predicted outcomes: a case study on homelessness services","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Kube","year":"2019"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1214\/009053607000000811","article-title":"Rodeo: sparse, greedy nonparametric regression","volume":"36","author":"Lafferty","year":"2008","journal-title":"The Annals of Statistics"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"2389","DOI":"10.1214\/19-AOAS1282","article-title":"Propensity score weighting for causal inference with multiple treatments","volume":"13","author":"Li","year":"2019","journal-title":"The Annals of Applied Statistics"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1111\/rssb.12066","article-title":"Variance function partially linear single-index models","volume":"77","author":"Lian","year":"2015","journal-title":"Journal of the Royal Statistical Society: Series B (Statistical Methodology)"},{"key":"2023022408360763100_","article-title":"A semiparametric approach to model effect modification","author":"Liang","year":"2020","journal-title":"Journal of the American Statistical Association"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"2272","DOI":"10.1214\/009053606000000722","article-title":"Component selection and smoothing in multivariate nonpara-metric regression","volume":"34","author":"Lin","year":"2006","journal-title":"The Annals of Statistics"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"3776","DOI":"10.1002\/sim.7844","article-title":"Augmented outcome-weighted learning for estimating optimal dynamic treatment regimens","volume":"37","author":"Liu","year":"2018","journal-title":"Statistics in Medicine"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"493","DOI":"10.1177\/0962280211428383","article-title":"Variable selection for optimal treatment decision","volume":"22","author":"Lu","year":"2013","journal-title":"Statistical Methods in Medical Research"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"382","DOI":"10.1214\/13-AOS1195","article-title":"On efficient dimension reduction with respect to a statistical functional of interest","volume":"42","author":"Luo","year":"2014","journal-title":"The Annals of Statistics"},{"key":"2023022408360763100_","first-page":"567","article-title":"Semiparametric estimation and inference of variance function with large dimensional covariates","volume":"29","author":"Ma","year":"2019","journal-title":"Statistica Sinica"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1093\/biomet\/93.1.75","article-title":"Efficient semiparametric estimator for heteroscedastic partially linear models","volume":"93","author":"Ma","year":"2006","journal-title":"Biometrika"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"1221","DOI":"10.1111\/j.1468-0262.2004.00530.x","article-title":"Statistical treatment rules for heterogeneous populations","volume":"72","author":"Manski","year":"2004","journal-title":"Econometrica"},{"key":"2023022408360763100_","article-title":"A robust method for estimating individualized treatment effect","volume-title":"arXiv preprint arXiv:2004.10108","author":"Meng","year":"2020"},{"key":"2023022408360763100_","first-page":"1","article-title":"Near-optimal individualized treatment recommendations","volume":"21","author":"Meng","year":"2020","journal-title":"Journal of Machine Learning Research"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","DOI":"10.1002\/9781118445112.stat08302","article-title":"Supervised learning","volume-title":"Wiley StatsRef: Statistics Reference Online","author":"Mo","year":"2021"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1111\/1467-9868.00389","article-title":"Optimal dynamic treatment regimes","volume":"65","author":"Murphy","year":"2003","journal-title":"Journal of the Royal Statistical Society: Series B (Statistical Methodology)"},{"key":"2023022408360763100_","first-page":"1073","article-title":"A generalization error for Q-learning","volume":"6","author":"Murphy","year":"2005","journal-title":"Journal of Machine Learning Research"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1007\/s10107-012-0629-5","article-title":"Gradient methods for minimizing composite functions","volume":"140","author":"Nesterov","year":"2013","journal-title":"Mathematical Programming"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"1349","DOI":"10.2307\/2951752","article-title":"The asymptotic variance of semiparametric estimators","volume":"62","author":"Newey","year":"1994","journal-title":"Econometrica"},{"key":"2023022408360763100_","first-page":"3","article-title":"GLP-1 receptor agonists: an alternative for rapid-acting insulin","volume":"41","author":"Ostroff","year":"2016","journal-title":"U.S. Pharmacist"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1080\/01621459.2020.1725522","article-title":"Improved doubly robust estimation in learning optimal individualized treatment rules","volume":"116","author":"Pan","year":"2021","journal-title":"Journal of the American Statistical Association"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"3601","DOI":"10.1214\/18-EJS1480","article-title":"D-learning to estimate optimal individual treatment rules","volume":"12","author":"Qi","year":"2018","journal-title":"Electronic Journal of Statistics"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"678","DOI":"10.1080\/01621459.2018.1529597","article-title":"Multi-armed angle-based direct learning for estimating optimal individualized treatment rules with various outcomes","volume":"115","author":"Qi","year":"2020","journal-title":"Journal of the American Statistical Association"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"1180","DOI":"10.1214\/10-AOS864","article-title":"Performance guarantees for individualized treatment rules","volume":"39","author":"Qian","year":"2011","journal-title":"The Annals of Statistics"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4419-9076-1_11","article-title":"Optimal structural nested models for optimal sequential decisions","volume-title":"Proceedings of the Second Seattle Symposium in Biostatistics","author":"Robins","year":"2004"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"846","DOI":"10.1080\/01621459.1994.10476818","article-title":"Estimation of regression coefficients when some regressors are not always observed","volume":"89","author":"Robins","year":"1994","journal-title":"Journal of the American Statistical Association"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1080\/01621459.1995.10476493","article-title":"Analysis of semiparametric regression models for repeated outcomes in the presence of missing data","volume":"90","author":"Robins","year":"1995","journal-title":"Journal of the American Statistical Association"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"439","DOI":"10.1093\/biomet\/ass013","article-title":"Improved double-robust estimation in missing data and causal inference models","volume":"99","author":"Rotnitzky","year":"2012","journal-title":"Biometrika"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"688","DOI":"10.1037\/h0037350","article-title":"Estimating causal effects of treatments in randomized and nonrandomized studies","volume":"66","author":"Rubin","year":"1974","journal-title":"Journal of Educational Psychology"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"925","DOI":"10.1214\/17-AOS1570","article-title":"High-dimensional A-learning for optimal dynamic treatment regimes","volume":"46","author":"Shi","year":"2018","journal-title":"The Annals of Statistics"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"661","DOI":"10.1093\/biomet\/asq035","article-title":"Bounded, efficient and doubly robust estimation with inverse weighting","volume":"97","author":"Tan","year":"2010","journal-title":"Biometrika"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"1517","DOI":"10.1080\/01621459.2014.951443","article-title":"A simple method for estimating interactions between a treatment and a large number of covariates","volume":"109","author":"Tian","year":"2014","journal-title":"Journal of the American Statistical Association"},{"key":"2023022408360763100_","volume-title":"Semiparametric theory and missing data","author":"Tsiatis","year":"2007"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4757-2545-2","volume-title":"Weak convergence and empirical processes: with applications to statistics","author":"van der Vaart","year":"1996"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"1024","DOI":"10.1080\/01621459.2014.958155","article-title":"Bias-reduced doubly robust estimation","volume":"110","author":"Vermeulen","year":"2015","journal-title":"Journal of the American Statistical Association"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"636","DOI":"10.1111\/biom.12306","article-title":"Doubly-robust dynamic treatment regimen estimation via weighted least squares","volume":"71","author":"Wallace","year":"2015","journal-title":"Biometrics"},{"key":"2023022408360763100_","article-title":"Learning from delayed rewards","author":"Watkins","year":"1989"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"2059","DOI":"10.1002\/sim.8102","article-title":"Robust regression for optimal individualized treatment rules","volume":"38","author":"Xiao","year":"2019","journal-title":"Statistics in Medicine"},{"key":"2023022408360763100_","article-title":"Multicategory angle-based learning for estimating optimal dynamic treatment regimes with censored data","author":"Xue","year":"2021","journal-title":"Journal of the American Statistical Association"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"625","DOI":"10.1093\/biomet\/asu017","article-title":"Multicategory angle-based large-margin classification","volume":"101","author":"Zhang","year":"2014","journal-title":"Biometrika"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"1099","DOI":"10.1198\/jasa.2011.tm10281","article-title":"Linear or nonlinear? Automatic structure discovery for partially linear models","volume":"106","author":"Zhang","year":"2011","journal-title":"Journal of the American Statistical Association"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"1010","DOI":"10.1111\/j.1541-0420.2012.01763.x","article-title":"A robust method for estimating optimal treatment regimes","volume":"68","author":"Zhang","year":"2012","journal-title":"Biometrics"},{"key":"2023022408360763100_","first-page":"1857","article-title":"Multicategory outcome weighted margin-based learning for estimating individualized treatment rules","volume":"30","author":"Zhang","year":"2020","journal-title":"Statistica Sinica"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"1106","DOI":"10.1080\/01621459.2012.695674","article-title":"Estimating individualized treatment rules using outcome weighted learning","volume":"107","author":"Zhao","year":"2012","journal-title":"Journal of the American Statistical Association"},{"key":"2023022408360763100_","first-page":"1","article-title":"Efficient augmentation and relaxation learning for individualized treatment rules using observational data","volume":"20","author":"Zhao","year":"2019","journal-title":"Journal of Machine Learning Research"},{"key":"2023022408360763100_","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1080\/01621459.2015.1093947","article-title":"Residual weighted learning for estimating individualized treatment rules","volume":"112","author":"Zhou","year":"2017","journal-title":"Journal of the American Statistical Association"},{"key":"2023022408360763100_","article-title":"Offline multi-action policy learning: generalization and optimization","volume-title":"arXiv preprint arXiv:1810.04778","author":"Zhou","year":"2018"},{"key":"2023022408360763100_","first-page":"869","article-title":"Dimension reduction for conditional variance in regressions","volume":"19","author":"Zhu","year":"2009","journal-title":"Statistica Sinica"},{"key":"2023022408360763100_","first-page":"1235","article-title":"Semiparametric estimation of conditional heteroscedasticity via single-index modeling","volume":"23","author":"Zhu","year":"2013","journal-title":"Statistica Sinica"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/rssb.12474","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1111\/rssb.12474","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/84\/2\/440\/49322506\/jrsssb_84_2_440.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/84\/2\/440\/49322506\/jrsssb_84_2_440.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,14]],"date-time":"2023-11-14T04:37:02Z","timestamp":1699936622000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/article\/84\/2\/440\/7056158"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,14]]},"references-count":70,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2021,12,14]]},"published-print":{"date-parts":[[2022,4,1]]}},"URL":"https:\/\/doi.org\/10.1111\/rssb.12474","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,4]]},"published":{"date-parts":[[2021,12,14]]}}}