{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,12]],"date-time":"2026-02-12T14:06:48Z","timestamp":1770905208959,"version":"3.50.1"},"reference-count":59,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2022,12,6]],"date-time":"2022-12-06T00:00:00Z","timestamp":1670284800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000006","name":"Office of Naval Research","doi-asserted-by":"publisher","award":["N00014-17-1-2141"],"award-info":[{"award-number":["N00014-17-1-2141"]}],"id":[{"id":"10.13039\/100000006","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000006","name":"Office of Naval Research","doi-asserted-by":"publisher","award":["R305D200019"],"award-info":[{"award-number":["R305D200019"]}],"id":[{"id":"10.13039\/100000006","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000006","name":"Office of Naval Research","doi-asserted-by":"publisher","award":["2051246"],"award-info":[{"award-number":["2051246"]}],"id":[{"id":"10.13039\/100000006","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000006","name":"Office of Naval Research","doi-asserted-by":"publisher","award":["2153019"],"award-info":[{"award-number":["2153019"]}],"id":[{"id":"10.13039\/100000006","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100005246","name":"Institute of Education Sciences","doi-asserted-by":"publisher","award":["N00014-17-1-2141"],"award-info":[{"award-number":["N00014-17-1-2141"]}],"id":[{"id":"10.13039\/100005246","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100005246","name":"Institute of Education Sciences","doi-asserted-by":"publisher","award":["R305D200019"],"award-info":[{"award-number":["R305D200019"]}],"id":[{"id":"10.13039\/100005246","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100005246","name":"Institute of Education Sciences","doi-asserted-by":"publisher","award":["2051246"],"award-info":[{"award-number":["2051246"]}],"id":[{"id":"10.13039\/100005246","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100005246","name":"Institute of Education Sciences","doi-asserted-by":"publisher","award":["2153019"],"award-info":[{"award-number":["2153019"]}],"id":[{"id":"10.13039\/100005246","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Science Foundation","award":["N00014-17-1-2141"],"award-info":[{"award-number":["N00014-17-1-2141"]}]},{"name":"National Science Foundation","award":["R305D200019"],"award-info":[{"award-number":["R305D200019"]}]},{"name":"National Science Foundation","award":["2051246"],"award-info":[{"award-number":["2051246"]}]},{"name":"National Science Foundation","award":["2153019"],"award-info":[{"award-number":["2153019"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>A wide range of machine-learning-based approaches have been developed in the past decade, increasing our ability to accurately model nonlinear and nonadditive response surfaces. This has improved performance for inferential tasks such as estimating average treatment effects in situations where standard parametric models may not fit the data well. These methods have also shown promise for the related task of identifying heterogeneous treatment effects. However, the estimation of both overall and heterogeneous treatment effects can be hampered when data are structured within groups if we fail to correctly model the dependence between observations. Most machine learning methods do not readily accommodate such structure. This paper introduces a new algorithm, stan4bart, that combines the flexibility of Bayesian Additive Regression Trees (BART) for fitting nonlinear response surfaces with the computational and statistical efficiencies of using Stan for the parametric components of the model. We demonstrate how stan4bart can be used to estimate average, subgroup, and individual-level treatment effects with stronger performance than other flexible approaches that ignore the multilevel structure of the data as well as multilevel approaches that have strict parametric forms.<\/jats:p>","DOI":"10.3390\/e24121782","type":"journal-article","created":{"date-parts":[[2022,12,7]],"date-time":"2022-12-07T02:18:48Z","timestamp":1670379528000},"page":"1782","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Stan and BART for Causal Inference: Estimating Heterogeneous Treatment Effects Using the Power of Stan and the Flexibility of Machine Learning"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9576-3064","authenticated-orcid":false,"given":"Vincent","family":"Dorie","sequence":"first","affiliation":[{"name":"Code for America, San Francisco, CA 94103, USA"}]},{"given":"George","family":"Perrett","sequence":"additional","affiliation":[{"name":"Department of Applied Statistics, Social Science, and the Humanities, New York University, New York, NY 10003, USA"}]},{"given":"Jennifer L.","family":"Hill","sequence":"additional","affiliation":[{"name":"Department of Applied Statistics, Social Science, and the Humanities, New York University, New York, NY 10003, USA"}]},{"given":"Benjamin","family":"Goodrich","sequence":"additional","affiliation":[{"name":"Department of Political Science, Columbia University, New York, NY 10025, USA"}]}],"member":"1968","published-online":{"date-parts":[[2022,12,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1198\/jcgs.2010.08162","article-title":"Bayesian nonparametric modeling for causal inference","volume":"20","author":"Hill","year":"2011","journal-title":"J. Comput. Graph. Stat."},{"key":"ref_2","unstructured":"LeDell, E. (h2oEnsemble: H2O Ensemble Learning, 2016). h2oEnsemble: H2O Ensemble Learning, R Package Version 0.1.8."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1228","DOI":"10.1080\/01621459.2017.1319839","article-title":"Estimation and Inference of Heterogeneous Treatment Effects using Random Forests","volume":"113","author":"Wager","year":"2018","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"4156","DOI":"10.1073\/pnas.1804597116","article-title":"Metalearners for estimating heterogeneous treatment effects using machine learning","volume":"116","author":"Sekhon","year":"2019","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"532","DOI":"10.1177\/0962280217729845","article-title":"Scalable collaborative targeted learning for high-dimensional data","volume":"28","author":"Ju","year":"2019","journal-title":"Stat. Methods Med. Res."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1989","DOI":"10.1214\/19-AOAS1266","article-title":"A Semiparametric Modeling Approach Using Bayesian Additive Regression Trees with an Application to Evaluate Heterogeneous Treatment Effects","volume":"13","author":"Zeldow","year":"2019","journal-title":"Ann. Appl. Stat."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"965","DOI":"10.1214\/19-BA1195","article-title":"Bayesian Regression Tree Models for Causal Inference: Regularization, Confounding, and Heterogeneous Effects (with Discussion)","volume":"15","author":"Hahn","year":"2020","journal-title":"Bayesian Anal."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1198\/073500102288618702","article-title":"Was There a Riverside Miracle? A Hierarchical Framework for Evaluating Programs with Grouped Data","volume":"21","author":"Dehejia","year":"2003","journal-title":"J. Bus. Econ. Stat."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Gelman, A., and Hill, J. (2007). Data Analysis Using Regression and Multilevel\/Hierarchical Models, Cambridge University Press.","DOI":"10.32614\/CRAN.package.arm"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Hill, J. (2013). The SAGE Handbook of Multilevel Modeling, SAGE. Chapter Multilevel Models and Causal Inference.","DOI":"10.4135\/9781446247600.n12"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1214\/12-AOAS583","article-title":"Agnostic notes on regression adjustments to experimental data: Reexamining Freedman\u2019s critique","volume":"7","author":"Lin","year":"2013","journal-title":"Ann. Appl. Stat."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Sch\u00f6lkopf, B., Platt, J., and Hoffman, T. (2007). Bayesian Ensemble Learning. Advances in Neural Information Processing Systems 19, MIT Press.","DOI":"10.7551\/mitpress\/7503.001.0001"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"266","DOI":"10.1214\/09-AOAS285","article-title":"BART: Bayesian Additive Regression Trees","volume":"4","author":"Chipman","year":"2010","journal-title":"Ann. Appl. Stat."},{"key":"ref_14","unstructured":"Dorie, V. (dbarts: Discrete Bayesian Additive Regression Trees Sampler, 2022). dbarts: Discrete Bayesian Additive Regression Trees Sampler, R Package Version 0.9-22."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1214\/18-STS667","article-title":"Automated versus do-it-yourself methods for causal inference: Lessons learned from a data analysis competition","volume":"34","author":"Dorie","year":"2019","journal-title":"Stat. Sci."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1093\/bioinformatics\/btq660","article-title":"Bayesian ensemble methods for survival prediction in gene expression data","volume":"27","author":"Bonato","year":"2010","journal-title":"Bioinformatics"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"405","DOI":"10.1080\/10618600.2019.1677243","article-title":"Heteroscedastic BART using multiplicative regression trees","volume":"29","author":"Pratola","year":"2020","journal-title":"J. Comput. Graph. Stat."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1111\/biom.13107","article-title":"Semiparametric mixed-scale models using shared Bayesian forests","volume":"76","author":"Linero","year":"2020","journal-title":"Biometrics"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1108\/S0731-90532019000040B006","article-title":"Fully nonparametric Bayesian additive regression trees","volume":"Volume 40","author":"George","year":"2019","journal-title":"Topics in Identification, Limited Dependent Variables, Partial Observability, Experimentation, and Flexible Modeling: Part B"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"756","DOI":"10.1080\/01621459.2020.1813587","article-title":"Log-Linear Bayesian Additive Regression Trees for Multinomial Logistic and Count Regression Models","volume":"116","author":"Murray","year":"2021","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1080\/00273171.2011.570161","article-title":"Challenges with Propensity Score Strategies in a High-Dimensional Setting and a Potential Alternative","volume":"46","author":"Hill","year":"2011","journal-title":"Multivar. Behav. Res."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1386","DOI":"10.1214\/13-AOAS630","article-title":"Assessing lack of common support in causal inference using Bayesian nonparametrics: Implications for evaluating the effect of breastfeeding on children\u2019s cognitive outcomes","volume":"7","author":"Hill","year":"2013","journal-title":"Ann. Appl. Stat."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"3453","DOI":"10.1002\/sim.6973","article-title":"A flexible, interpretable framework for assessing sensitivity to unmeasured confounding","volume":"35","author":"Dorie","year":"2016","journal-title":"Stat. Med."},{"key":"ref_24","first-page":"103","article-title":"Assessing methods for generalizing experimental impact estimates to target samples","volume":"9","author":"Kern","year":"2016","journal-title":"J. Res. Educ. Eff."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"3309","DOI":"10.1002\/sim.7820","article-title":"Comparing methods for estimation of heterogeneous treatment effects using observational data from health care databases","volume":"37","author":"Wendling","year":"2018","journal-title":"Stat. Med."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v097.i01","article-title":"Nonparametric Machine Learning and Efficient Computation with Bayesian Additive Regression Trees: The BART R Package","volume":"97","author":"Sparapani","year":"2021","journal-title":"J. Stat. Softw."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1060","DOI":"10.1017\/S0003055419000480","article-title":"BARP: Improving Mister P Using Bayesian Additive Regression Trees","volume":"113","author":"Bisbee","year":"2019","journal-title":"Am. Political Sci. Rev."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"364","DOI":"10.1038\/s41586-019-1466-y","article-title":"A national experiment reveals where a growth mindset improves achievement","volume":"573","author":"Yeager","year":"2019","journal-title":"Nature"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"512","DOI":"10.1038\/s41586-022-04907-7","article-title":"A synergistic mindsets intervention protects adolescents from stress","volume":"607","author":"Yeager","year":"2022","journal-title":"Nature"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1177\/09567976211028984","article-title":"Teacher Mindsets Help Explain Where a Growth-Mindset Intervention Does and Doesn\u2019t Work","volume":"33","author":"Yeager","year":"2022","journal-title":"Psychol. Sci."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"310","DOI":"10.1007\/s11336-021-09805-x","article-title":"Robust Machine Learning for Treatment Effects in Multilevel Observational Studies Under Cluster-level Unmeasured Confounding","volume":"87","author":"Suk","year":"2022","journal-title":"Psychometrika"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2665","DOI":"10.1002\/sim.8924","article-title":"Nonparametric machine learning for precision medicine with longitudinal clinical trials and Bayesian additive regression trees with mixed models","volume":"40","author":"Spanbauer","year":"2021","journal-title":"Stat. Med."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"557","DOI":"10.4310\/SII.2018.v11.n4.a1","article-title":"Predicting human-driving behavior to help driverless vehicles drive: Random intercept Bayesian additive regression trees","volume":"11","author":"Tan","year":"2018","journal-title":"Stat. Its Interface"},{"key":"ref_34","first-page":"318","article-title":"Using Multivariate Matched Sampling and Regression Adjustment to Control Bias in Observational Studies","volume":"74","author":"Rubin","year":"1979","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"945","DOI":"10.1080\/01621459.1986.10478354","article-title":"Statistics and Causal Inference","volume":"81","author":"Holland","year":"1986","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_36","unstructured":"Vegetabile, B.G. (2021). On the Distinction Between \u201cConditional Average Treatment Effects\u201d (CATE) and \u201cIndividual Treatment Effects\u201d (ITE) Under Ignorability Assumptions. arXiv."},{"key":"ref_37","first-page":"491","article-title":"Examining treatment effect heterogeneity using BART","volume":"76","author":"Carnegie","year":"2019","journal-title":"Obs. Stud."},{"key":"ref_38","first-page":"395","article-title":"Assessing sensitivity to unmeasured confounding using a simulated potential confounder","volume":"9","author":"Carnegie","year":"2016","journal-title":"J. Res. Educ. Eff."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1214\/aos\/1176344064","article-title":"Bayesian Inference for Causal Effects: The role of randomization","volume":"6","author":"Rubin","year":"1978","journal-title":"Ann. Stat."},{"key":"ref_40","unstructured":"Team, S.D. (2022, August 14). Stan Modeling Language Users Guide and Reference Manual; Version 2.29. Available online: https:\/\/mc-stan.org\/docs\/2_29\/stan-users-guide\/."},{"key":"ref_41","first-page":"1593","article-title":"The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo","volume":"15","author":"Hoffman","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Betancourt, M. (2017). A conceptual introduction to Hamiltonian Monte Carlo. arXiv.","DOI":"10.3150\/16-BEJ810"},{"key":"ref_43","first-page":"2","article-title":"MCMC using Hamiltonian dynamics","volume":"2","author":"Neal","year":"2011","journal-title":"Handb. Markov Chain. Monte Carlo"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v067.i01","article-title":"Fitting Linear Mixed-Effects Models Using lme4","volume":"67","author":"Bates","year":"2015","journal-title":"J. Stat. Softw."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1989","DOI":"10.1016\/j.jmva.2009.04.008","article-title":"Generating random correlation matrices based on vines and extended onion method","volume":"100","author":"Lewandowski","year":"2009","journal-title":"J. Multivar. Anal."},{"key":"ref_46","unstructured":"Bates, D., Kliegl, R., Vasishth, S., and Baayen, H. (2015). Parsimonious Mixed Models. arXiv."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1750","DOI":"10.1214\/14-AOAS755","article-title":"Variable selection for BART: An application to gene regulation","volume":"8","author":"Bleich","year":"2014","journal-title":"Ann. Appl. Stat."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1080\/00031305.1992.10475878","article-title":"Explaining the Gibbs Sampler","volume":"46","author":"Casella","year":"1992","journal-title":"Am. Stat."},{"key":"ref_49","unstructured":"Stan Development Team (RStan: The R Interface to Stan, 2022). RStan: The R Interface to Stan, R Package Version 2.21.5."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"5048","DOI":"10.1002\/sim.8347","article-title":"Bayesian additive regression trees and the General BART model","volume":"38","author":"Tan","year":"2019","journal-title":"Stat. Med."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3236009","article-title":"A Survey of Methods for Explaining Black Box Models","volume":"51","author":"Guidotti","year":"2018","journal-title":"ACM Comput. Surv."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"1264","DOI":"10.1080\/01621459.1999.10473879","article-title":"Parameter Expansion for Data Augmentation","volume":"94","author":"Liu","year":"1999","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1093\/biomet\/86.2.301","article-title":"Seeking efficient data augmentation schemes via conditional and marginal augmentation","volume":"86","author":"Meng","year":"1999","journal-title":"Biometrika"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1198\/106186008X287337","article-title":"Using Redundant Parameterizations to Fit Hierarchical Models","volume":"17","author":"Gelman","year":"2008","journal-title":"J. Comput. Graph. Stat."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1214\/18-STS682","article-title":"Contributions of Model Features to BART Causal Inference Performance Using ACIC 2016 Competition Data","volume":"34","author":"Carnegie","year":"2019","journal-title":"Stat. Sci."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1093\/pan\/mpw015","article-title":"Bias Amplification and Bias Unmasking","volume":"24","author":"Middleton","year":"2016","journal-title":"Political Anal."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1353\/obs.2018.0016","article-title":"Potential for Bias Inflation with Grouped Data: A Comparison of Estimators and a Sensitivity Analysis Strategy","volume":"4","author":"Scott","year":"2018","journal-title":"Obs. Stud."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Infant Health and Development Program (1990). Enhancing the outcomes of low-birth-weight, premature infants. J. Am. Med Assoc., 22, 3035\u20133042.","DOI":"10.1001\/jama.1990.03440220059030"},{"key":"ref_59","first-page":"350","article-title":"Effects of early intervention on cognitive function of low birth weight preterm infants","volume":"120","author":"Liaw","year":"1991","journal-title":"J. Pediatr."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/12\/1782\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:34:59Z","timestamp":1760146499000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/12\/1782"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,6]]},"references-count":59,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["e24121782"],"URL":"https:\/\/doi.org\/10.3390\/e24121782","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,6]]}}}