{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T17:53:56Z","timestamp":1791482036336,"version":"4.3.4"},"reference-count":47,"publisher":"Oxford University Press (OUP)","issue":"12","license":[{"start":{"date-parts":[[2024,6,14]],"date-time":"2024-06-14T00:00:00Z","timestamp":1718323200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100000026","name":"National Institute on Drug Abuse","doi-asserted-by":"publisher","award":["R01 DA056407"],"award-info":[{"award-number":["R01 DA056407"]}],"id":[{"id":"10.13039\/100000026","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,12,2]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Investigators often report estimates of the average treatment effect (ATE). While the ATE summarizes the effect of a treatment on average, it does not provide any information about the effect of treatment within any individual. A treatment strategy that uses an individual\u2019s information to tailor treatment to maximize benefit is known as an optimal dynamic treatment rule (ODTR). Treatment, however, is typically not limited to a single point in time; consequently, learning an optimal rule for a time-varying treatment may involve not just learning the extent to which the comparative treatments\u2019 benefits vary across the characteristics of individuals, but also learning the extent to which the comparative treatments\u2019 benefits vary as relevant circumstances evolve within an individual. The goal of this paper is to provide a tutorial for estimating ODTR from longitudinal observational and clinical trial data for applied researchers. We describe an approach that uses a doubly robust unbiased transformation of the conditional ATE. We then learn a time-varying ODTR for when to increase buprenorphine-naloxone dose to minimize a return to regular opioid use among patients with opioid use disorder. Our analysis highlights the utility of ODTRs in the context of sequential decision-making: The learned ODTR outperforms a clinically defined strategy.<\/jats:p>\n               <jats:p>This article is part of a Special Collection on Pharmacoepidemiology.<\/jats:p>","DOI":"10.1093\/aje\/kwae122","type":"journal-article","created":{"date-parts":[[2024,6,16]],"date-time":"2024-06-16T04:45:35Z","timestamp":1718513135000},"page":"1768-1775","source":"Crossref","is-referenced-by-count":7,"title":["Learning optimal dynamic treatment regimes from longitudinal data"],"prefix":"10.1093","volume":"193","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1378-4831","authenticated-orcid":false,"given":"Nicholas T","family":"Williams","sequence":"first","affiliation":[{"name":"Columbia University Department of Epidemiology, Mailman School of Public Health, , New York, NY 10032,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3392-3541","authenticated-orcid":false,"given":"Katherine L","family":"Hoffman","sequence":"additional","affiliation":[{"name":"Columbia University Department of Epidemiology, Mailman School of Public Health, , New York, NY 10032,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Iv\u00e1n","family":"D\u00edaz","sequence":"additional","affiliation":[{"name":"New York University Division of Biostatistics, Department of Population Health Sciences, Grossman School of Medicine, , New York, NY 10016,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9417-7960","authenticated-orcid":false,"given":"Kara E","family":"Rudolph","sequence":"additional","affiliation":[{"name":"Columbia University Department of Epidemiology, Mailman School of Public Health, , New York, NY 10032,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2024,6,14]]},"reference":[{"issue":"533","key":"2024121223371463400_ref1","doi-asserted-by":"publisher","first-page":"159","DOI":"10.1080\/01621459.2020.1863224","article-title":"Introduction to the theory and methods special issue on precision medicine and individualized policy discovery","volume":"116","author":"Kosorok","year":"2021","journal-title":"J Am Stat Assoc."},{"issue":"1","key":"2024121223371463400_ref2","doi-asserted-by":"publisher","first-page":"263","DOI":"10.1146\/annurev-statistics-030718-105251","article-title":"Precision medicine","volume":"6","author":"Kosorok","year":"2019","journal-title":"Annu Rev Stat Appl"},{"issue":"2","key":"2024121223371463400_ref3","doi-asserted-by":"publisher","first-page":"331","DOI":"10.1111\/1467-9868.00389","article-title":"Optimal dynamic treatment regimes","volume":"65","author":"Murphy","year":"2003","journal-title":"J R Stat Soc Ser B Stat Methodol."},{"key":"2024121223371463400_ref4","doi-asserted-by":"publisher","DOI":"10.1016\/j.drugalcdep.2021.109031","article-title":"Optimizing opioid use disorder treatment with naltrexone or buprenorphine","volume":"228","author":"Rudolph","year":"2021","journal-title":"Drug Alcohol Depend."},{"issue":"3","key":"2024121223371463400_ref5","doi-asserted-by":"publisher","first-page":"681","DOI":"10.1093\/biomet\/ast014","article-title":"Robust estimation of optimal dynamic treatment regimes for sequential treatment decisions","volume":"100","author":"Zhang","year":"2013","journal-title":"Biometrika."},{"issue":"1","key":"2024121223371463400_ref6","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1515\/ijb-2015-0052","article-title":"Super-learning of an optimal dynamic treatment rule","volume":"12","author":"Luedtke","year":"2016","journal-title":"Int J Biostat."},{"issue":"3","key":"2024121223371463400_ref7","doi-asserted-by":"publisher","first-page":"723","DOI":"10.1093\/biomet\/asy017","article-title":"Targeted learning ensembles for optimal individualized treatment rules with time-to-event outcomes","volume":"105","author":"D\u00edaz","year":"2018","journal-title":"Biometrika."},{"issue":"510","key":"2024121223371463400_ref8","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1080\/01621459.2014.937488","article-title":"New statistical learning methods for estimating optimal dynamic treatment regimes","volume":"110","author":"Zhao","year":"2015","journal-title":"J Am Stat Assoc."},{"key":"2024121223371463400_ref9","doi-asserted-by":"publisher","DOI":"10.1016\/j.drugalcdep.2022.109609","article-title":"Buprenorphine & methadone dosing strategies to reduce risk of relapse in the treatment of opioid use disorder","volume":"239","author":"Rudolph","year":"2022","journal-title":"Drug Alcohol Depend."},{"key":"2024121223371463400_ref10","first-page":"249","volume-title":"Causal Inference: What If","author":"Hern\u00e1n","year":"2020"},{"issue":"2","key":"2024121223371463400_ref11","doi-asserted-by":"publisher","first-page":"18","DOI":"10.2202\/1557-4679.1212","article-title":"When to start treatment? A systematic approach to the comparison of dynamic regimes using observational data","volume":"6","author":"Cain","year":"2010","journal-title":"Int J Biostat."},{"key":"2024121223371463400_ref12","volume-title":"Learning From Delayed Rewards","author":"Watkins","year":"1989"},{"issue":"4","key":"2024121223371463400_ref13","doi-asserted-by":"publisher","first-page":"629","DOI":"10.1002\/cjs.11162","article-title":"Q-learning for estimating optimal dynamic treatment rules from observational data","volume":"40","author":"Moodie","year":"2012","journal-title":"Can J Stat."},{"issue":"499","key":"2024121223371463400_ref14","doi-asserted-by":"publisher","first-page":"1106","DOI":"10.1080\/01621459.2012.695674","article-title":"Estimating individualized treatment rules using outcome weighted learning","volume":"107","author":"Zhao","year":"2012","journal-title":"J Am Stat Assoc."},{"issue":"517","key":"2024121223371463400_ref15","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1080\/01621459.2015.1093947","article-title":"Residual weighted learning for estimating individualized treatment rules","volume":"112","author":"Zhou","year":"2017","journal-title":"J Am Stat Assoc."},{"issue":"26","key":"2024121223371463400_ref16","doi-asserted-by":"publisher","first-page":"3776","DOI":"10.1002\/sim.7844","article-title":"Augmented outcome-weighted learning for estimating optimal dynamic treatment regimens","volume":"37","author":"Liu","year":"2018","journal-title":"Stat Med."},{"key":"2024121223371463400_ref17","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1007\/978-1-4419-9076-1_11","volume-title":"Proceedings of the Second Seattle Symposium in Biostatistics: Analysis of Correlated Data","author":"Robins","year":"2004"},{"issue":"1","key":"2024121223371463400_ref18","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1515\/jci-2013-0022","article-title":"Targeted learning of the mean outcome under an optimal dynamic treatment rule","volume":"3","author":"Laan","year":"2015","journal-title":"J Causal Inference."},{"issue":"1","key":"2024121223371463400_ref19","doi-asserted-by":"publisher","first-page":"2","DOI":"10.2202\/1557-4679.1008","article-title":"Statistical inference for variable importance","volume":"2","author":"Laan","year":"2006","journal-title":"Int J Biostat."},{"key":"2024121223371463400_ref20","article-title":"Towards optimal doubly robust estimation of heterogeneous causal effects","author":"Kennedy","year":"21, 2023","journal-title":"arXiv"},{"issue":"427","key":"2024121223371463400_ref21","doi-asserted-by":"publisher","first-page":"846","DOI":"10.1080\/01621459.1994.10476818","article-title":"Estimation of regression coefficients when some regressors are not always observed","volume":"89","author":"Robins","year":"1994","journal-title":"J Am Stat Assoc."},{"key":"2024121223371463400_ref22","author":"Williams","year":"2023"},{"issue":"1","key":"2024121223371463400_ref23","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1515\/1557-4679.1370","article-title":"Targeted minimum loss based estimation of causal effects of multiple time point interventions","volume":"8","author":"Laan","year":"2012","journal-title":"Int J Biostat."},{"key":"2024121223371463400_ref24","volume-title":"arXiv","author":"Luedtke","year":"17, 2018"},{"key":"2024121223371463400_ref25","doi-asserted-by":"crossref","first-page":"459","DOI":"10.1007\/978-1-4419-9782-1","volume-title":"Targeted Learning: Causal Inference for Observational and Experimental Data","author":"Laan","year":"2011"},{"issue":"6","key":"2024121223371463400_ref26","doi-asserted-by":"publisher","first-page":"470","DOI":"10.1097\/ADM.0000000000000164","article-title":"Patterns and quality of buprenorphine opioid agonist treatment in a large Medicaid program","volume":"9","author":"Gordon","year":"2015","journal-title":"J Addict Med."},{"key":"2024121223371463400_ref27","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1016\/j.jsat.2018.10.006","article-title":"Evidence-based treatment for opioid use disorders: a national study of methadone dose levels, 2011\u20132017","volume":"96","author":"D\u2019Aunno","year":"2019","journal-title":"J Subst Abuse Treat."},{"issue":"4","key":"2024121223371463400_ref28","doi-asserted-by":"publisher","first-page":"605","DOI":"10.15288\/jsad.2013.74.605","article-title":"Buprenorphine\/naloxone and methadone maintenance treatment outcomes for opioid analgesic, heroin, and combined users: findings from Starting Treatment with Agonist Replacement Therapies (START)","volume":"74","author":"Potter","year":"2013","journal-title":"J Stud Alcohol Drugs."},{"issue":"1-2","key":"2024121223371463400_ref29","doi-asserted-by":"publisher","first-page":"71","DOI":"10.1016\/j.drugalcdep.2012.08.002","article-title":"Buprenorphine\/naloxone and methadone effects on laboratory indices of liver health: a randomized trial","volume":"1280","author":"Saxon","year":"2013","journal-title":"Drug Alcohol Depend."},{"issue":"2","key":"2024121223371463400_ref30","doi-asserted-by":"publisher","first-page":"189","DOI":"10.1016\/j.cct.2010.01.003","article-title":"A multi-site, two-phase, Prescription Opioid Addiction Treatment Study (POATS): rationale, design, and methodology","volume":"310","author":"Weiss","year":"2010","journal-title":"Contemp Clin Trials."},{"issue":"10118","key":"2024121223371463400_ref31","doi-asserted-by":"publisher","first-page":"309","DOI":"10.1016\/S0140-6736(17)32812-X","article-title":"Comparative effectiveness of extended-release naltrexone versus buprenorphine-naloxone for opioid relapse prevention (X:BOT): a multicentre, open-label, randomised controlled trial","volume":"391","author":"Lee","year":"2018","journal-title":"Lancet."},{"issue":"12","key":"2024121223371463400_ref32","doi-asserted-by":"publisher","first-page":"1238","DOI":"10.1001\/archgenpsychiatry.2011.121","article-title":"Adjunctive counseling during brief and extended buprenorphine-naloxone treatment for prescription opioid dependence: a 2-phase randomized controlled trial","volume":"680","author":"Weiss","year":"2011","journal-title":"Arch Gen Psychiatry."},{"issue":"5","key":"2024121223371463400_ref33","doi-asserted-by":"publisher","first-page":"748","DOI":"10.1093\/aje\/kwac217","article-title":"Optimally choosing medication type for patients with opioid use disorder","volume":"192","author":"Rudolph","year":"2022","journal-title":"Am J Epidemiol."},{"issue":"1","key":"2024121223371463400_ref34","doi-asserted-by":"publisher","first-page":"25","DOI":"10.2202\/1544-6115.1309","article-title":"Super learner","volume":"6","author":"Laan","year":"2007","journal-title":"Stat Appl Genet Mol Biol."},{"issue":"1","key":"2024121223371463400_ref35","doi-asserted-by":"publisher","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression shrinkage and selection via the lasso","volume":"58","author":"Tibshirani","year":"1996","journal-title":"J R Stat Soc B Methodol."},{"issue":"1","key":"2024121223371463400_ref36","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1214\/aos\/1176347963","article-title":"Multivariate adaptive regression splines","volume":"19","author":"Friedman","year":"1991","journal-title":"Ann Stat."},{"key":"2024121223371463400_ref37","volume-title":"lightgbm: Light Gradient Boosting Machine","author":"Shi","year":"2022"},{"issue":"1","key":"2024121223371463400_ref38","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1007\/BF00117832","article-title":"Stacked regressions","volume":"24","author":"Breiman","year":"1996","journal-title":"Mach Learn."},{"issue":"4","key":"2024121223371463400_ref39","doi-asserted-by":"publisher","first-page":"1276","DOI":"10.1093\/ije\/dyad023","article-title":"Practical considerations for specifying a super learner","volume":"52","author":"Phillips","year":"2023","journal-title":"Int J Epidemiol."},{"key":"2024121223371463400_ref40","author":"R Core Team","year":"2022"},{"issue":"2","key":"2024121223371463400_ref41","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1353\/obs.2023.0019","article-title":"lmtp: an R package for estimating the causal effects of modified treatment policies","volume":"9","author":"Williams","year":"2023","journal-title":"Obs Stud"},{"issue":"542","key":"2024121223371463400_ref42","doi-asserted-by":"publisher","first-page":"846","DOI":"10.1080\/01621459.2021.1955691","article-title":"Non-parametric causal effects based on longitudinal modified treatment policies","volume":"118","author":"D\u00edaz","year":"2023","journal-title":"J Am Stat Assoc."},{"issue":"1","key":"2024121223371463400_ref43","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1515\/ijb-2020-0127","article-title":"The optimal dynamic treatment rule superlearner: considerations, performance, and application to criminal justice interventions","volume":"19","author":"Montoya","year":"2022","journal-title":"Int J Biostat."},{"key":"2024121223371463400_ref44","volume-title":"Medications for Opioid Use Disorder for Healthcare and Addiction Professionals, Policymakers, Patients, and Families. Updated 2021. Treatment Improvement Protocol TIP 63","author":"Substance Abuse and Mental Health Services Administration","year":"2021"},{"key":"2024121223371463400_ref45","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1109\/ICHI.2016.13","volume-title":"2016 IEEE International Conference on Healthcare Informatics (ICHI), Chicago, IL, USA","author":"Wang","year":"2016"},{"key":"2024121223371463400_ref46","volume-title":"Targeted Learning in R: Causal Data Science With the tlverse Software Ecosystem","author":"Laan","year":"7, 2023"},{"issue":"1","key":"2024121223371463400_ref47","doi-asserted-by":"publisher","first-page":"283","DOI":"10.1515\/ijb-2015-0007","article-title":"Optimal individualized treatments in resource-limited settings","volume":"12","author":"Luedtke","year":"2016","journal-title":"Int J Biostat."}],"container-title":["American Journal of Epidemiology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/aje\/advance-article-pdf\/doi\/10.1093\/aje\/kwae122\/59335088\/kwae122.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/aje\/article-pdf\/193\/12\/1768\/60935861\/kwae122.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/aje\/article-pdf\/193\/12\/1768\/60935861\/kwae122.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,12]],"date-time":"2024-12-12T23:37:34Z","timestamp":1734046654000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/aje\/article\/193\/12\/1768\/7693607"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,14]]},"references-count":47,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2024,6,14]]},"published-print":{"date-parts":[[2024,12,2]]}},"URL":"https:\/\/doi.org\/10.1093\/aje\/kwae122","relation":{},"ISSN":["0002-9262","1476-6256"],"issn-type":[{"value":"0002-9262","type":"print"},{"value":"1476-6256","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,12]]},"published":{"date-parts":[[2024,6,14]]}}}