{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,27]],"date-time":"2026-10-27T18:48:00Z","timestamp":1793126880867,"version":"4.5.0"},"reference-count":37,"publisher":"Springer Science and Business Media LLC","issue":"S2","license":[{"start":{"date-parts":[[2020,10,26]],"date-time":"2020-10-26T00:00:00Z","timestamp":1603670400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,10,26]],"date-time":"2020-10-26T00:00:00Z","timestamp":1603670400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100006093","name":"Patient-Centered Outcomes Research Institute","doi-asserted-by":"publisher","award":["n\/a"],"award-info":[{"award-number":["n\/a"]}],"id":[{"id":"10.13039\/100006093","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J GEN INTERN MED"],"published-print":{"date-parts":[[2020,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n<jats:title>Introduction<\/jats:title>\n<jats:p>Complex health interventions (CHIs) are increasingly studied in comparative effectiveness research (CER), and there is a need for improvements in CHI research practices. The Patient-Centered Outcomes Research Institute (PCORI) Methodology Committee (MC) launched an effort in 2016 to develop formal guidance on this topic.<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Objective<\/jats:title>\n<jats:p>To develop a set of minimal standards for scientifically valid, transparent, and reproducible CER studies of CHIs. The standards are intended to apply to research examining a broad range of healthcare interventions including delivery system, behavior change, and other non-pharmacological interventions.<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Methods<\/jats:title>\n<jats:p>We conducted a literature review, reviewed existing methods guidance, and developed standards through an iterative process involving the MC, two panels of external research methods experts, and a 60-day public comment period. The final standards were approved by the PCORI MC and adopted by the PCORI Board of Governors on April 30, 2018.<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Results<\/jats:title>\n<jats:p>The final standards include the following: (1) fully describe the intervention and comparator and define their core functions, (2) specify the hypothesized causal pathways and their theoretical basis, (3) specify how adaptations to the form of the intervention and comparator will be allowed and recorded, (4) plan and\u00a0describe a process evaluation, and (5) select patient outcomes informed by the causal pathway.<\/jats:p>\n<\/jats:sec><jats:sec>\n<jats:title>Discussion<\/jats:title>\n<jats:p>The new standards offer three major contributions to research: (1) they provide a simple framework to help investigators address the major methodological features of a CHI study, (2) they emphasize the importance of the causal model and the need to understand <jats:italic>how<\/jats:italic> a CHI achieves its effects rather than simply measuring these effects, and (3) they require description of a CHI using the concepts of core functions and forms. While these standards apply formally to PCORI-funded CER studies, they have broad applicability.<\/jats:p>\n<\/jats:sec>","DOI":"10.1007\/s11606-020-06093-6","type":"journal-article","created":{"date-parts":[[2020,10,26]],"date-time":"2020-10-26T18:02:54Z","timestamp":1603735374000},"page":"875-881","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":48,"title":["Improving Comparative Effectiveness Research of Complex Health Interventions: Standards from the Patient-Centered Outcomes Research Institute (PCORI)"],"prefix":"10.1007","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3684-614X","authenticated-orcid":false,"given":"Laura C.","family":"Esmail","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rebecca","family":"Barasky","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Brian S.","family":"Mittman","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David H.","family":"Hickam","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2020,10,26]]},"reference":[{"key":"6093_CR1","doi-asserted-by":"publisher","first-page":"a1655","DOI":"10.1136\/bmj.a1655","volume":"337","author":"P Craig","year":"2008","unstructured":"Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ (Clin Res Ed). 2008;337:a1655. https:\/\/doi.org\/10.1136\/bmj.a1655","journal-title":"BMJ (Clin Res Ed)"},{"key":"6093_CR2","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1016\/j.jclinepi.2017.06.011","volume":"90","author":"JM Guise","year":"2017","unstructured":"Guise JM, Chang C, Butler M, Viswanathan M, Tugwell P. AHRQ series on complex intervention systematic reviews\u2014paper 1: an introduction to a series of articles that provide guidance and tools for reviews of complex interventions. J Clin Epidemiol 2017;90:6-10. https:\/\/doi.org\/10.1016\/j.jclinepi.2017.06.011","journal-title":"J Clin Epidemiol"},{"key":"6093_CR3","volume-title":"The PCORI Methodology Report","author":"The PCORI Methodology Committee","year":"2019","unstructured":"The PCORI Methodology Committee. The PCORI Methodology Report 2019."},{"issue":"15","key":"6093_CR4","doi-asserted-by":"publisher","first-page":"1636","DOI":"10.1001\/jama.2012.466","volume":"307","author":"The Methodology Committee of the Patient-Centered Outcomes Research Institute (PCORI)","year":"2012","unstructured":"The Methodology Committee of the Patient-Centered Outcomes Research Institute (PCORI). Methodological standards and patient-centeredness in comparative effectiveness research: the PCORI perspective. JAMA. 2012;307(15):1636-40. https:\/\/doi.org\/10.1001\/jama.2012.466","journal-title":"JAMA."},{"issue":"2","key":"6093_CR5","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1136\/jech-2013-202869","volume":"68","author":"G Moore","year":"2014","unstructured":"Moore G, Audrey S, Barker M, Bond L, Bonell C, Cooper C, et al. Process evaluation in complex public health intervention studies: the need for guidance. J Epidemiol Community Health 2014;68(2):101-2. https:\/\/doi.org\/10.1136\/jech-2013-202869","journal-title":"J Epidemiol Community Health"},{"key":"6093_CR6","doi-asserted-by":"crossref","unstructured":"Richards DA, Hallberg IR. Complex Interventions in Health: an Overview of Research Methods: Routledge; 2015.","DOI":"10.4324\/9780203794982"},{"key":"6093_CR7","unstructured":"PCORI. Methodology Committee. 2019. https:\/\/www.pcori.org\/about-us\/governance\/methodology-committee. Accessed 1 Sept 2019."},{"key":"6093_CR8","unstructured":"The PCORI Methodology Committee. Appendix B: Response to public comment. The PCORI Methodology Report 2019. p. 73-82."},{"key":"6093_CR9","unstructured":"PCORI. PCORI Methodology Standards. 2019. https:\/\/www.pcori.org\/research-results\/about-our-research\/research-methodology\/pcori-methodology-standards. Accessed 1 Sept 2019."},{"key":"6093_CR10","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1186\/1748-5908-4-40","volume":"4","author":"S Michie","year":"2009","unstructured":"Michie S, Fixsen D, Grimshaw JM, Eccles MP. Specifying and reporting complex behaviour change interventions: the need for a scientific method. Implement Sci IS 2009;4:40. https:\/\/doi.org\/10.1186\/1748-5908-4-40","journal-title":"Implement Sci IS"},{"key":"6093_CR11","doi-asserted-by":"publisher","first-page":"g1687","DOI":"10.1136\/bmj.g1687","volume":"348","author":"TC Hoffmann","year":"2014","unstructured":"Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: Template for Intervention Description and Replication (TIDieR) checklist and guide. BMJ (Clin Res Ed). 2014;348:g1687. https:\/\/doi.org\/10.1136\/bmj.g1687","journal-title":"BMJ (Clin Res Ed)"},{"key":"6093_CR12","doi-asserted-by":"publisher","first-page":"204","DOI":"10.1186\/s13063-015-0709-y","volume":"16","author":"R Mohler","year":"2015","unstructured":"Mohler R, Kopke S, Meyer G. Criteria for Reporting the Development and Evaluation of Complex Interventions in healthcare: revised guideline (CReDECI 2). Trials. 2015;16:204. https:\/\/doi.org\/10.1186\/s13063-015-0709-y","journal-title":"Trials."},{"key":"6093_CR13","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1016\/j.jclinepi.2017.06.017","volume":"90","author":"JM Guise","year":"2017","unstructured":"Guise JM, Butler M, Chang C, Viswanathan M, Pigott T, Tugwell P. AHRQ series on complex intervention systematic reviews\u2014paper 7: PRISMA-CI elaboration and explanation. J Clin Epidemiol 2017;90:51-8. https:\/\/doi.org\/10.1016\/j.jclinepi.2017.06.017","journal-title":"J Clin Epidemiol"},{"issue":"7455","key":"6093_CR14","doi-asserted-by":"publisher","first-page":"1561","DOI":"10.1136\/bmj.328.7455.1561","volume":"328","author":"P Hawe","year":"2004","unstructured":"Hawe P, Shiell A, Riley T. Complex interventions: how \u201cout of control\u201d can a randomised controlled trial be? BMJ (Clin Res Ed) 2004;328(7455):1561-3. https:\/\/doi.org\/10.1136\/bmj.328.7455.1561","journal-title":"BMJ (Clin Res Ed)"},{"key":"6093_CR15","doi-asserted-by":"publisher","first-page":"307","DOI":"10.1146\/annurev-publhealth-031912-114421","volume":"36","author":"P Hawe","year":"2015","unstructured":"Hawe P. Lessons from complex interventions to improve health. Annu Rev Public Health 2015;36:307-23. https:\/\/doi.org\/10.1146\/annurev-publhealth-031912-114421","journal-title":"Annu Rev Public Health"},{"issue":"6","key":"6093_CR16","doi-asserted-by":"publisher","first-page":"1032","DOI":"10.1007\/s11606-018-4818-7","volume":"34","author":"M Perez Jolles","year":"2019","unstructured":"Perez Jolles M, Lengnick-Hall R, Mittman BS. Core functions and forms of complex health interventions: a patient-centered medical home illustration. J Gen Intern Med 2019;34(6):1032-8. https:\/\/doi.org\/10.1007\/s11606-018-4818-7","journal-title":"J Gen Intern Med"},{"key":"6093_CR17","doi-asserted-by":"publisher","unstructured":"Hawe P, Shiell A, Riley T. In response to Spillane V., Byrne M.C., Byrne M., Leathem C.S., O\u2019Malley M. & Cupples M.E. (2007) Monitoring treatment fidelity in a randomized trial of a complex intervention. J Adv Nurs 60(3), 343-352. Important considerations for standardizing complex interventions. J Adv Nurs. 2008;62(2):267. https:\/\/doi.org\/10.1111\/j.1365-2648.2008.04686.x","DOI":"10.1111\/j.1365-2648.2008.04686.x"},{"issue":"9","key":"6093_CR18","doi-asserted-by":"publisher","first-page":"1031","DOI":"10.1007\/s00038-016-0809-9","volume":"61","author":"M Villeval","year":"2016","unstructured":"Villeval M, Bidault E, Shoveller J, Alias F, Basson JC, Frasse C, et al. Enabling the transferability of complex interventions: exploring the combination of an intervention\u2019s key functions and implementation. Int J Public Health 2016;61(9):1031-8. https:\/\/doi.org\/10.1007\/s00038-016-0809-9","journal-title":"Int J Public Health"},{"issue":"1","key":"6093_CR19","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1186\/s13012-019-0880-8","volume":"14","author":"M Villeval","year":"2019","unstructured":"Villeval M, Gaborit E, Berault F, Lang T, Kelly-Irving M. Do the key functions of an intervention designed from the same specifications vary according to context? Investigating the transferability of a public health intervention in France. Implement Sci IS 2019;14(1):35. https:\/\/doi.org\/10.1186\/s13012-019-0880-8","journal-title":"Implement Sci IS"},{"key":"6093_CR20","doi-asserted-by":"publisher","unstructured":"McGuire AB, Powell KG, Treitler PC, Wagner KD, Smith KP, Cooperman N, et al. Emergency department-based peer support for opioid use disorder: emergent functions and forms. J Subst Abus Treat 2019. https:\/\/doi.org\/10.1016\/j.jsat.2019.06.013","DOI":"10.1016\/j.jsat.2019.06.013"},{"issue":"1","key":"6093_CR21","doi-asserted-by":"publisher","first-page":"274","DOI":"10.1186\/1472-6963-8-274","volume":"8","author":"R Byng","year":"2008","unstructured":"Byng R, Norman I, Redfern S, Jones R. Exposing the key functions of a complex intervention for shared care in mental health: case study of a process evaluation. BMC Health Serv Res 2008;8(1):274.","journal-title":"BMC Health Serv Res"},{"key":"6093_CR22","doi-asserted-by":"crossref","unstructured":"Hill J, Cuthel AM, Lin P, Grudzen CR. Primary Palliative Care for Emergency Medicine (PRIM-ER): applying form and function to a theory-based complex intervention. Contemp Clin Trials Commun 2020:100570.","DOI":"10.1016\/j.conctc.2020.100570"},{"key":"6093_CR23","doi-asserted-by":"publisher","first-page":"h1258","DOI":"10.1136\/bmj.h1258","volume":"350","author":"GF Moore","year":"2015","unstructured":"Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ (Clin Res Ed). 2015;350:h1258. https:\/\/doi.org\/10.1136\/bmj.h1258","journal-title":"BMJ (Clin Res Ed)"},{"key":"6093_CR24","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1016\/j.jclinepi.2017.06.013","volume":"90","author":"M Butler","year":"2017","unstructured":"Butler M, Epstein RA, Totten A, Whitlock EP, Ansari MT, Damschroder LJ, et al. AHRQ series on complex intervention systematic reviews-paper 3: adapting frameworks to develop protocols. J Clin Epidemiol 2017;90:19-27. https:\/\/doi.org\/10.1016\/j.jclinepi.2017.06.013","journal-title":"J Clin Epidemiol"},{"key":"6093_CR25","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1186\/1748-5908-4-50","volume":"4","author":"LJ Damschroder","year":"2009","unstructured":"Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci IS 2009;4:50. https:\/\/doi.org\/10.1186\/1748-5908-4-50","journal-title":"Implement Sci IS"},{"key":"6093_CR26","doi-asserted-by":"crossref","unstructured":"Newhouse RP, Himmelfarb CD, Morlock L, Frick KD, Pronovost P, Liang Y. A phased cluster-randomized trial of rural hospitals testing a quality collaborative to improve heart failure care: organizational context matters. Med Care 2013:396-403.","DOI":"10.1097\/MLR.0b013e318286e32e"},{"issue":"4","key":"6093_CR27","doi-asserted-by":"publisher","first-page":"581","DOI":"10.1111\/j.0887-378X.2004.00325.x","volume":"82","author":"T Greenhalgh","year":"2004","unstructured":"Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 2004;82(4):581-629. https:\/\/doi.org\/10.1111\/j.0887-378X.2004.00325.x","journal-title":"Milbank Q"},{"issue":"1","key":"6093_CR28","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1186\/s13012-019-0898-y","volume":"14","author":"SW Stirman","year":"2019","unstructured":"Stirman SW, Baumann AA, Miller CJ. The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions. Implement Sci 2019;14(1):58.","journal-title":"Implement Sci"},{"issue":"4","key":"6093_CR29","doi-asserted-by":"publisher","first-page":"619","DOI":"10.1007\/BF00937995","volume":"19","author":"LJ Bauman","year":"1991","unstructured":"Bauman LJ, Stein RE, Ireys HT. Reinventing fidelity: the transfer of social technology among settings. Am J Community Psychol 1991;19(4):619-39.","journal-title":"Am J Community Psychol"},{"issue":"5","key":"6093_CR30","doi-asserted-by":"publisher","first-page":"331","DOI":"10.1016\/j.jamda.2011.01.006","volume":"12","author":"MF Reelick","year":"2011","unstructured":"Reelick MF, Faes MC, Esselink RA, Kessels RP, Olde Rikkert MG. How to perform a preplanned process evaluation for complex interventions in geriatric medicine: exemplified with the process evaluation of a complex falls-prevention program for community-dwelling frail older fallers. J Am Med Dir Assoc 2011;12(5):331-6. https:\/\/doi.org\/10.1016\/j.jamda.2011.01.006","journal-title":"J Am Med Dir Assoc"},{"key":"6093_CR31","doi-asserted-by":"crossref","unstructured":"Craig P, Di Ruggiero E, Frolich KL, Mykhalovskiy E, White M, Campbell R, et al. Taking account of context in population health intervention research: guidance for producers, users and funders of research. 2018.","DOI":"10.3310\/CIHR-NIHR-01"},{"key":"6093_CR32","unstructured":"AcademyHealth. Evaluating Complex Health Interventions: A Guide to Rigorous Research Designs. 2017."},{"issue":"1","key":"6093_CR33","doi-asserted-by":"publisher","first-page":"92","DOI":"10.1186\/s12874-019-0736-6","volume":"19","author":"L Minary","year":"2019","unstructured":"Minary L, Trompette J, Kivits J, Cambon L, Tarquinio C, Alla F. Which design to evaluate complex interventions? Toward a methodological framework through a systematic review. BMC Med Res Methodol 2019;19(1):92. https:\/\/doi.org\/10.1186\/s12874-019-0736-6","journal-title":"BMC Med Res Methodol"},{"issue":"3","key":"6093_CR34","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1177\/0962280209105014","volume":"19","author":"R Emsley","year":"2010","unstructured":"Emsley R, Dunn G, White IR. Mediation and moderation of treatment effects in randomised controlled trials of complex interventions. Stat Methods Med Res 2010;19(3):237-70. https:\/\/doi.org\/10.1177\/0962280209105014","journal-title":"Stat Methods Med Res"},{"issue":"6","key":"6093_CR35","doi-asserted-by":"publisher","first-page":"e002889","DOI":"10.1136\/bmjopen-2013-002889","volume":"3","author":"A O\u2019Cathain","year":"2013","unstructured":"O\u2019Cathain A, Thomas K, Drabble S, Rudolph A, Hewison J. What can qualitative research do for randomised controlled trials? A systematic mapping review. BMJ Open 2013;3(6):e002889.","journal-title":"BMJ Open"},{"key":"6093_CR36","doi-asserted-by":"publisher","first-page":"b3496","DOI":"10.1136\/bmj.b3496","volume":"339","author":"S Lewin","year":"2009","unstructured":"Lewin S, Glenton C, Oxman AD. Use of qualitative methods alongside randomised controlled trials of complex healthcare interventions: methodological study. BMJ (Clin Res Ed.). 2009;339:b3496.","journal-title":"BMJ (Clin Res Ed.)"},{"key":"6093_CR37","unstructured":"Craig P, Matthews L, Moore L, Simpson S, Skivington K. Updated Guidance: Developing and Evaluating Complex Interventions [Draft for Consultation]. 2019."}],"container-title":["Journal of General Internal Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11606-020-06093-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11606-020-06093-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11606-020-06093-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,10,26]],"date-time":"2021-10-26T10:56:31Z","timestamp":1635245791000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11606-020-06093-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,26]]},"references-count":37,"journal-issue":{"issue":"S2","published-print":{"date-parts":[[2020,11]]}},"alternative-id":["6093"],"URL":"https:\/\/doi.org\/10.1007\/s11606-020-06093-6","relation":{},"ISSN":["0884-8734","1525-1497"],"issn-type":[{"value":"0884-8734","type":"print"},{"value":"1525-1497","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,10,26]]},"assertion":[{"value":"3 September 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 July 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"26 October 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Compliance with Ethical Standards"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of Interest"}},{"value":"The views presented in this article are solely the responsibility of the authors and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute (PCORI), its Board of Governors, or its Methodology Committee.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Disclaimer"}}]}}