{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,3]],"date-time":"2026-11-03T22:17:10Z","timestamp":1793744230708,"version":"4.6.1"},"reference-count":59,"publisher":"Wiley","issue":"6","license":[{"start":{"date-parts":[[2019,10,10]],"date-time":"2019-10-10T00:00:00Z","timestamp":1570665600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Health Services Research"],"published-print":{"date-parts":[[2019,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Objective<\/jats:title><jats:p>To demonstrate the performance of methodologies that include machine learning (ML) algorithms to estimate average treatment effects under the assumption of exogeneity (selection on observables).<\/jats:p><\/jats:sec><jats:sec><jats:title>Data Sources<\/jats:title><jats:p>Simulated data and observational data on hospitalized adults.<\/jats:p><\/jats:sec><jats:sec><jats:title>Study Design<\/jats:title><jats:p>We assessed the performance of several ML\u2010based estimators, including Targeted Maximum Likelihood Estimation, Bayesian Additive Regression Trees, Causal Random Forests, Double Machine Learning, and Bayesian Causal Forests, applying these methods to simulated data as well as data on the effects of right heart catheterization.<\/jats:p><\/jats:sec><jats:sec><jats:title>Principal Findings<\/jats:title><jats:p>In Monte Carlo studies, ML\u2010based estimators generated estimates with smaller bias than traditional regression approaches, demonstrating substantial (69 percent\u201098 percent) bias reduction in some scenarios. Bayesian Causal Forests and Double Machine Learning were top performers, although all were sensitive to high dimensional (&gt;150) sets of covariates.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>ML\u2010based methods are promising methods for estimating treatment effects, allowing for the inclusion of many covariates and automating the search for nonlinearities and interactions among variables. We provide guidance and sample code for researchers interested in implementing these tools in their own empirical work.<\/jats:p><\/jats:sec>","DOI":"10.1111\/1475-6773.13212","type":"journal-article","created":{"date-parts":[[2019,10,11]],"date-time":"2019-10-11T05:09:27Z","timestamp":1570770567000},"page":"1273-1282","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":42,"title":["Estimating treatment effects with machine learning"],"prefix":"10.1111","volume":"54","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1280-3196","authenticated-orcid":false,"given":"K. John","family":"McConnell","sequence":"first","affiliation":[{"name":"Center for Health Systems Effectiveness Oregon Health &amp; Science University Portland Oregon"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9577-6287","authenticated-orcid":false,"given":"Stephan","family":"Lindner","sequence":"additional","affiliation":[{"name":"Center for Health Systems Effectiveness Oregon Health &amp; Science University Portland Oregon"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2019,10,10]]},"reference":[{"key":"e_1_2_10_2_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.aar5045"},{"key":"e_1_2_10_3_1","doi-asserted-by":"publisher","DOI":"10.1111\/1475-6773.12848"},{"key":"e_1_2_10_4_1","doi-asserted-by":"publisher","DOI":"10.1093\/biostatistics\/kxx012"},{"key":"e_1_2_10_5_1","doi-asserted-by":"publisher","DOI":"10.1162\/003465304323023651"},{"key":"e_1_2_10_6_1","doi-asserted-by":"publisher","DOI":"10.1001\/jama.1996.03540110043030"},{"key":"e_1_2_10_7_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1020371312283"},{"key":"e_1_2_10_8_1","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1353\/obs.2018.0018","article-title":"Estimating the variance of a propensity score matching estimator for the average treatment effect","volume":"4","author":"Pingel R","year":"2018","journal-title":"Observ Stud"},{"key":"e_1_2_10_9_1","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.2016.1260466"},{"key":"e_1_2_10_10_1","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/asn055"},{"key":"e_1_2_10_11_1","doi-asserted-by":"publisher","DOI":"10.1007\/s12561-011-9036-3"},{"key":"e_1_2_10_12_1","doi-asserted-by":"publisher","DOI":"10.1257\/aer.98.2.345"},{"key":"e_1_2_10_13_1","doi-asserted-by":"publisher","DOI":"10.1257\/aer.98.2.351"},{"key":"e_1_2_10_14_1","doi-asserted-by":"publisher","DOI":"10.1257\/aer.98.2.357"},{"key":"e_1_2_10_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jeconom.2012.01.001"},{"key":"e_1_2_10_16_1","doi-asserted-by":"publisher","DOI":"10.2307\/2533848"},{"key":"e_1_2_10_17_1","doi-asserted-by":"publisher","DOI":"10.1198\/jcgs.2011.09219"},{"key":"e_1_2_10_18_1","first-page":"507","volume-title":"The Economics of Artificial Intelligence: An Agenda","author":"Athey S","year":"2018"},{"key":"e_1_2_10_19_1","doi-asserted-by":"publisher","DOI":"10.1257\/jep.31.2.87"},{"key":"e_1_2_10_20_1","doi-asserted-by":"publisher","DOI":"10.1097\/MLR.0b013e3181de9e17"},{"key":"e_1_2_10_21_1","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMp1606181"},{"key":"e_1_2_10_22_1","doi-asserted-by":"publisher","DOI":"10.1093\/cid\/cix731"},{"key":"e_1_2_10_23_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.csbj.2014.11.005"},{"key":"e_1_2_10_24_1","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9781316576533"},{"key":"e_1_2_10_25_1","doi-asserted-by":"publisher","DOI":"10.1111\/1475-6773.12464"},{"key":"e_1_2_10_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-009-9124-7"},{"key":"e_1_2_10_27_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4419-9782-1_3"},{"key":"e_1_2_10_28_1","doi-asserted-by":"publisher","DOI":"10.1093\/aje\/kws241"},{"key":"e_1_2_10_29_1","doi-asserted-by":"publisher","DOI":"10.1257\/aer.p20151023"},{"key":"e_1_2_10_30_1","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/70.1.41"},{"issue":"1","key":"e_1_2_10_31_1","first-page":"1","article-title":"Targeted maximum likelihood learning","volume":"2","author":"Laan MJ","year":"2006","journal-title":"Int J Biostat"},{"key":"e_1_2_10_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4419-9782-1"},{"key":"e_1_2_10_33_1","doi-asserted-by":"publisher","DOI":"10.1198\/jcgs.2010.08162"},{"key":"e_1_2_10_34_1","doi-asserted-by":"publisher","DOI":"10.1214\/18-AOS1709"},{"key":"e_1_2_10_35_1","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.2017.1319839"},{"key":"e_1_2_10_36_1","doi-asserted-by":"publisher","DOI":"10.1257\/aer.p20171038"},{"key":"e_1_2_10_37_1","unstructured":"HahnPR MurrayJS CarvalhoCM.Bayesian regression tree models for causal inference: regularization confounding and heterogeneous effects (Mimeo).2018.https:\/\/math.la.asu.edu\/~prhahn\/BCF.pdf. Accessed November 1 2018."},{"key":"e_1_2_10_38_1","doi-asserted-by":"publisher","DOI":"10.1093\/aje\/kww165"},{"key":"e_1_2_10_39_1","doi-asserted-by":"publisher","DOI":"10.1002\/sim.7628"},{"key":"e_1_2_10_40_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.stamet.2005.02.003"},{"key":"e_1_2_10_41_1","doi-asserted-by":"publisher","DOI":"10.2202\/1557-4679.1115"},{"issue":"1","key":"e_1_2_10_42_1","first-page":"17","article-title":"Estimation based on case\u2010control designs with known prevalence probability","volume":"4","author":"Laan MJ","year":"2008","journal-title":"Int J Biostat"},{"key":"e_1_2_10_43_1","doi-asserted-by":"publisher","DOI":"10.2307\/2988216"},{"issue":"340","key":"e_1_2_10_44_1","first-page":"768","article-title":"A modal search technique for predictive nominal scale multivariate analysis","volume":"67","author":"Messenger R","year":"1972","journal-title":"J Am Stat Assoc"},{"key":"e_1_2_10_45_1","doi-asserted-by":"publisher","DOI":"10.1214\/09-AOAS285"},{"key":"e_1_2_10_46_1","doi-asserted-by":"publisher","DOI":"10.1214\/16-BA1044"},{"key":"e_1_2_10_47_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1010933404324"},{"key":"e_1_2_10_48_1","doi-asserted-by":"publisher","DOI":"10.2307\/1907330"},{"key":"e_1_2_10_49_1","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1963.10480682"},{"key":"e_1_2_10_50_1","doi-asserted-by":"publisher","DOI":"10.2307\/1392377"},{"key":"e_1_2_10_51_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-84858-7"},{"key":"e_1_2_10_52_1","doi-asserted-by":"publisher","DOI":"10.3982\/ECTA9626"},{"key":"e_1_2_10_53_1","first-page":"1414","volume-title":"Proceedings of the 34th International Conference on Machine Learning \u2010 Volume 70","author":"Hartford J","year":"2017"},{"key":"e_1_2_10_54_1","doi-asserted-by":"publisher","DOI":"10.1086\/688177"},{"key":"e_1_2_10_55_1","first-page":"1","article-title":"On the use of the lasso for instrumental variables estimation with some invalid instruments","volume":"14","author":"Windmeijer F","year":"2018","journal-title":"J Am Stat Assoc"},{"key":"e_1_2_10_56_1","unstructured":"LuC NieX WagerS.Robust nonparametric difference\u2010in\u2010differences estimation. arXiv:190511622 [stat].2019.http:\/\/arxiv.org\/abs\/1905.11622. Accessed August 1 2019."},{"key":"e_1_2_10_57_1","doi-asserted-by":"crossref","unstructured":"AbrahamS SunL.Estimating dynamic treatment effects in event studies with heterogeneous treatment effects. Working Paper.2018.","DOI":"10.2139\/ssrn.3158747"},{"key":"e_1_2_10_58_1","doi-asserted-by":"crossref","unstructured":"ZhengW van derLaanMJ.Asymptotic theory for cross\u2010validated targeted maximum likelihood estimation(UC Berkeley Division of Biostatistics Working Paper 273).2010.","DOI":"10.2202\/1557-4679.1181"},{"key":"e_1_2_10_59_1","unstructured":"YangY ChengG DunsonD.Semiparametric Bernstein\u2010von Mises theorem: second order studies(ArXiv:1503.04493).2015.https:\/\/arxiv.org\/abs\/1503.04493. Accessed March 28 2019."},{"key":"e_1_2_10_60_1","doi-asserted-by":"publisher","DOI":"10.1111\/ectj.12097"}],"container-title":["Health Services Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/1475-6773.13212","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1111\/1475-6773.13212","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/1475-6773.13212","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,8]],"date-time":"2023-09-08T22:21:21Z","timestamp":1694211681000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/1475-6773.13212"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,10,10]]},"references-count":59,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2019,12]]}},"alternative-id":["10.1111\/1475-6773.13212"],"URL":"https:\/\/doi.org\/10.1111\/1475-6773.13212","archive":["Portico"],"relation":{},"ISSN":["0017-9124","1475-6773"],"issn-type":[{"value":"0017-9124","type":"print"},{"value":"1475-6773","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,10,10]]},"assertion":[{"value":"2019-10-10","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}