{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T11:20:37Z","timestamp":1778498437472,"version":"3.51.4"},"reference-count":34,"publisher":"Ovid Technologies (Wolters Kluwer Health)","issue":"6","content-domain":{"domain":["lww.com","ovid.com"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,11]]},"abstract":"<jats:sec>\n                    <jats:title>Background:<\/jats:title>\n                    <jats:p>The Super Learner is an ensemble learning method that has been widely used with doubly robust causal effect estimators. It is recommended to deploy the Super Learner with a diverse library of algorithms. To our knowledge, however, the magnitude of the improvements gained by including many algorithms has not yet been systematically evaluated in common epidemiologic research settings.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods:<\/jats:title>\n                    <jats:p>We applied Super Learning with two doubly robust estimators, augmented inverse probability weighting (AIPW) and targeted minimum loss-based estimation (TMLE), to estimate the average treatment effect (ATE) of high periconceptional dietary fruit and vegetable density on the risk of preeclampsia among 7,923 women from the nuMoM2b study. Using a reference ensemble with a diverse library of algorithms, we compared estimates under different sets of algorithms included in the Super Learner to evaluate whether ATE estimates were sensitive to library choices.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results:<\/jats:title>\n                    <jats:p>The doubly robust estimators fitted with the reference Super Learner ensemble suggested \u22652.5 cups\/1,000 kcal of total fruit and vegetable density was associated with a lower risk of preeclampsia. ATE estimated on the risk difference scale by AIPW was \u22120.019 (95% confidence interval = \u22120.036, \u22120.003) and by TMLE was \u22120.023 (95% confidence interval = \u22120.039, \u22120.007). Excluding any individual algorithm from the reference ensemble had little impact on estimates from either AIPW or TMLE. However, relying on a single algorithm (e.g., extreme gradient boosting) yielded results that were much more variable.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion:<\/jats:title>\n                    <jats:p>Our empirical findings support recommendations to build ensemble learners for doubly robust estimators using a diverse array of flexible machine learning algorithms.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1097\/ede.0000000000001906","type":"journal-article","created":{"date-parts":[[2025,8,15]],"date-time":"2025-08-15T12:00:46Z","timestamp":1755259246000},"page":"760-768","update-policy":"https:\/\/doi.org\/10.1097\/lww.0000000000001000","source":"Crossref","is-referenced-by-count":1,"title":["Algorithm Selection for Estimating Causal Effects: Nulliparous Pregnancy Outcomes Study: Monitoring Mothers to Be"],"prefix":"10.1097","volume":"36","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-6519-3219","authenticated-orcid":false,"given":"Zhaohua","family":"Zeng","sequence":"first","affiliation":[{"name":"Department of Epidemiology, Emory University, Atlanta, GA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lisa M.","family":"Bodnar","sequence":"additional","affiliation":[{"name":"Department of Epidemiology, University of Pittsburgh, Pittsburgh, PA."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1510-8175","authenticated-orcid":false,"given":"Ashley I.","family":"Naimi","sequence":"additional","affiliation":[{"name":"Department of Epidemiology, Emory University, Atlanta, GA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"276","published-online":{"date-parts":[[2025,8,15]]},"reference":[{"key":"R1-20260511","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1093\/pan\/mpp036","article-title":"An introduction to the augmented inverse propensity weighted estimator.","volume":"18","author":"Glynn","year":"2010","journal-title":"Polit Anal"},{"key":"R2-20260511","doi-asserted-by":"crossref","first-page":"Article 11","DOI":"10.2202\/1557-4679.1043","article-title":"Targeted maximum likelihood learning.","volume":"2","author":"van der Laan","year":"2006","journal-title":"Int J Biostat"},{"key":"R3-20260511","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1097\/EDE.0000000000001332","article-title":"Machine learning for causal inference: on the use of cross-fit estimators.","volume":"32","author":"Zivich","year":"2021","journal-title":"Epidemiology"},{"key":"R4-20260511","doi-asserted-by":"crossref","first-page":"1235","DOI":"10.1093\/ajcn\/nqaa027","article-title":"Machine learning as a strategy to account for dietary synergy: an illustration based on dietary intake and adverse pregnancy outcomes.","volume":"111","author":"Bodnar","year":"2020","journal-title":"Am J Clin Nutr"},{"key":"R5-20260511","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1001\/jamainternmed.2023.0258","article-title":"Comparative effectiveness of fludrocortisone and hydrocortisone vs hydrocortisone alone among patients with septic shock.","volume":"183","author":"Bosch","year":"2023","journal-title":"JAMA Intern Med"},{"key":"R6-20260511","doi-asserted-by":"crossref","first-page":"459","DOI":"10.1007\/s10654-018-0390-z","article-title":"Stacked generalization: an introduction to super learning.","volume":"33","author":"Naimi","year":"2018","journal-title":"Eur J Epidemiol"},{"key":"R7-20260511","doi-asserted-by":"crossref","first-page":"Article25","DOI":"10.2202\/1544-6115.1309","article-title":"Super learner.","volume":"6","author":"van der Laan","year":"2007","journal-title":"Stat Appl Genet Mol Biol"},{"key":"R8-20260511","first-page":"51","article-title":"Oracle inequalities for multi-fold cross validation.","volume":"24","author":"van der Vaart","year":"2006","journal-title":"Stat Decis"},{"key":"R11-20260511","first-page":"2825","article-title":"Scikit-learn: machine learning in Python.","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J Mach Learn Res"},{"key":"R12-20260511","doi-asserted-by":"crossref","first-page":"1545","DOI":"10.1093\/aje\/kwab200","article-title":"Demystifying statistical inference when using machine learning in causal research.","volume":"192","author":"Balzer","year":"2023","journal-title":"Am J Epidemiol"},{"key":"R13-20260511","doi-asserted-by":"crossref","first-page":"1276","DOI":"10.1093\/ije\/dyad023","article-title":"Practical considerations for specifying a super learner.","volume":"52","author":"Phillips","year":"2023","journal-title":"Int J Epidemiol"},{"key":"R14-20260511","doi-asserted-by":"crossref","first-page":"539.e1","DOI":"10.1016\/j.ajog.2015.01.019","article-title":"A description of the methods of the Nulliparous Pregnancy Outcomes Study: monitoring mothers-to-be (nuMoM2b).","volume":"212","author":"Haas","year":"2015","journal-title":"Am J Obstet Gynecol"},{"key":"R15-20260511","doi-asserted-by":"crossref","first-page":"680","DOI":"10.1016\/j.tjnut.2023.12.013","article-title":"Periconceptional dietary patterns and adverse pregnancy and birth outcomes.","volume":"154","author":"Bodnar","year":"2024","journal-title":"J Nutr"},{"key":"R16-20260511","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1016\/j.annepidem.2023.04.017","article-title":"Does heterogeneity underlie differences in treatment effects estimated from SuperLearner versus logistic regression? an application in nutritional epidemiology.","volume":"83","author":"Petersen","year":"2023","journal-title":"Ann Epidemiol"},{"key":"R17-20260511","doi-asserted-by":"crossref","first-page":"1327","DOI":"10.1016\/0895-4356(90)90099-B","article-title":"Validation of a self-administered diet history questionnaire using multiple diet records.","volume":"43","author":"Block","year":"1990","journal-title":"J Clin Epidemiol"},{"key":"R18-20260511","doi-asserted-by":"crossref","first-page":"1851","DOI":"10.1016\/j.ajog.2004.07.076","article-title":"Second trimester folate status and preterm birth.","volume":"191","author":"Siega-Riz","year":"2004","journal-title":"Am J Obstet Gynecol"},{"key":"R19-20260511","doi-asserted-by":"crossref","first-page":"686","DOI":"10.1016\/S0002-8223(21)00707-0","article-title":"Comparison of two dietary questionnaires validated against multiple dietary records collected during a 1-year period.","volume":"92","author":"Block","year":"1992","journal-title":"J Am Diet Assoc"},{"key":"R20-20260511","doi-asserted-by":"crossref","first-page":"856","DOI":"10.1093\/oxfordjournals.aje.a009203","article-title":"Associations of race\/ethnicity, education, and dietary intervention with the validity and reliability of a food frequency questionnaire: the Women\u2019s Health Trial Feasibility Study in minority populations.","volume":"146","author":"Kristal","year":"1997","journal-title":"Am J Epidemiol"},{"key":"R25-20260511","first-page":"1122","article-title":"Report of the American College of Obstetricians and gynecologists\u2019 task force on hypertension in pregnancy.","volume":"122","year":"2013","journal-title":"Obstet Gynecol"},{"key":"R26-20260511","doi-asserted-by":"crossref","first-page":"459","DOI":"10.1016\/j.ajcnut.2023.06.007","article-title":"Is the association between fruits and vegetables and preeclampsia due to higher dietary vitamin C and carotenoid intakes?","volume":"118","author":"Bodnar","year":"2023","journal-title":"Am J Clin Nutr"},{"key":"R27-20260511","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1071\/HE14050","article-title":"Developing a research and practice tool to measure walkability: a demonstration project.","volume":"25","author":"Giles-Corti","year":"2014","journal-title":"Health Promot J Austr"},{"key":"R28-20260511","doi-asserted-by":"crossref","first-page":"2456","DOI":"10.1056\/NEJMp1802313","article-title":"Making neighborhood-disadvantage metrics accessible\u2014the neighborhood atlas.","volume":"378","author":"Kind","year":"2018","journal-title":"N Engl J Med"},{"key":"R29-20260511","first-page":"433","article-title":"Rapid assessment of literacy levels of adult primary care patients.","volume":"23","author":"Davis","year":"1991","journal-title":"Fam Med"},{"key":"R30-20260511","doi-asserted-by":"crossref","first-page":"S228","DOI":"10.1067\/mob.2002.123054","article-title":"Motherisk-PUQE scoring system for nausea and vomiting of pregnancy.","volume":"186","author":"Koren","year":"2002","journal-title":"Am J Obstet Gynecol"},{"key":"R31-20260511","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/0165-0327(96)00008-0","article-title":"Validation of the Edinburgh Postnatal Depression Scale (EPDS) in non-postnatal women.","volume":"39","author":"Cox","year":"1996","journal-title":"J Affect Disord"},{"key":"R32-20260511","doi-asserted-by":"crossref","first-page":"385","DOI":"10.2307\/2136404","article-title":"A global measure of perceived stress.","volume":"24","author":"Cohen","year":"1983","journal-title":"J Health Soc Behav"},{"key":"R34-20260511","doi-asserted-by":"crossref","first-page":"1591","DOI":"10.1016\/j.jand.2018.05.021","article-title":"Update of the healthy eating index: HEI-2015.","volume":"118","author":"Krebs-Smith","year":"2018","journal-title":"J Acad Nutr Diet"},{"key":"R36-20260511","doi-asserted-by":"crossref","first-page":"685","DOI":"10.1093\/aje\/kwad012","article-title":"Defining and identifying average treatment effects.","volume":"192","author":"Naimi","year":"2023","journal-title":"Am J Epidemiol"},{"key":"R38-20260511","doi-asserted-by":"crossref","first-page":"C1","DOI":"10.1111\/ectj.12097","article-title":"Double\/debiased machine learning for treatment and structural parameters.","volume":"21","author":"Chernozhukov","year":"2018","journal-title":"Econom J"},{"key":"R39-20260511","doi-asserted-by":"crossref","first-page":"846","DOI":"10.1080\/01621459.1994.10476818","article-title":"Estimation of regression coefficients when some regressors are not always observed.","volume":"89","author":"Robins","year":"1994","journal-title":"J Am Stat Assoc"},{"key":"R41-20260511","article-title":"Semiparametric doubly robust targeted double machine learning: a review.","author":"Kennedy","year":"2022","journal-title":"arXiv preprint"},{"key":"R44-20260511","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1002\/(SICI)1097-0258(19970215)16:3<285::AID-SIM535>3.0.CO;2-#","article-title":"Toward a curse of dimensionality appropriate (CODA) asymptotic theory for semi-parametric models.","volume":"16","author":"Robins","year":"1997","journal-title":"Stat Med"},{"key":"R45-20260511","doi-asserted-by":"crossref","first-page":"292","DOI":"10.1080\/00031305.2021.2021984","article-title":"Demystifying statistical learning based on efficient influence functions.","volume":"76","author":"Hines","year":"2022","journal-title":"Am Stat"},{"key":"R46-20260511","article-title":"The fundamental limits of structure-agnostic functional estimation.","author":"Balakrishnan","year":"2023","journal-title":"arXiv preprint"}],"container-title":["Epidemiology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.lww.com\/10.1097\/EDE.0000000000001906","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,11]],"date-time":"2026-05-11T11:00:40Z","timestamp":1778497240000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.lww.com\/10.1097\/EDE.0000000000001906"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,15]]},"references-count":34,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2025]]}},"URL":"https:\/\/doi.org\/10.1097\/ede.0000000000001906","relation":{},"ISSN":["1044-3983","1531-5487"],"issn-type":[{"value":"1044-3983","type":"print"},{"value":"1531-5487","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,15]]}}}