{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"institution":[{"name":"Research Square"}],"indexed":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T11:16:12Z","timestamp":1770290172950,"version":"3.49.0"},"posted":{"date-parts":[[2024,8,9]]},"group-title":"In Review","reference-count":74,"publisher":"Springer Science and Business Media LLC","license":[{"start":{"date-parts":[[2024,8,9]],"date-time":"2024-08-09T00:00:00Z","timestamp":1723161600000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"accepted":{"date-parts":[[2024,6,21]]},"abstract":"<title>Abstract<\/title>\n        <p><bold>Background:<\/bold> Substance use disorders (SUDs) have a serious adverse impact on the physical and mental well-being of people with HIV. Previously, using a 39-site dual-randomized type 2 hybrid trial design, findings from the Substance Abuse Treatment to HIV Care Project supported the Implementation and Sustainment Facilitation (ISF) strategy to improve implementation and effectiveness of a motivational interviewing brief intervention (MIBI) for SUD within HIV service settings across the United States (US). Building on this trial, this parallel<bold> <\/bold>cluster-randomized type 3 hybrid trial aimed to test the incremental effectiveness of a pay-for-performance (P4P), a form of the \u201calter incentive\/allowance structures\u201d strategy.\n<bold>Methods:<\/bold> Twenty-six HIV service organizations, their staff participants (N=87), and their client participants (N=341) were cluster-randomized to one of two implementation conditions. The control condition included staff-focused training, feedback, and consultation (TFC) and team-focused implementation and sustainment (ISF). The experimental condition included TFC+ISF as well as P4P (TFC+ISF+P4P). P4P used financial incentives to reward MIBI implementation (US$10 per MIBI delivered) and MIBI implementation at or above a pre-defined level of quality (US$10 per demonstration). In addition to these outcomes, past 4-week changes\/reductions in client participant\u2019s days of primary substance use and anxiety symptoms were examined.\n<bold>Results:<\/bold> The addition of P4P had a large and significant effect on the number of MIBIs implemented (<italic>d<\/italic>=1.30, p&lt;.05) and reduction in anxiety (<italic>d=<\/italic>-1.54), but there was no impact on days of substance use. P4P had large effects on MIBI quality (<italic>d<\/italic>=1.24) and MIBI implementation effectiveness (<italic>d<\/italic>=1.28), but these were not significant (<italic>p<\/italic>&lt;.10).\n<bold>Conclusions:<\/bold> P4P is a form of the \u201calter incentive\/allowance structures\u201d strategy. Its function is to reward the implementation of a clinical innovation. Rewarding implementation is consistent with the theory of implementation effectiveness, which suggests implementation climate (i.e., the extent to which implementation is expected, supported, and rewarded) is a key antecedent of implementation effectiveness (i.e., the consistency and quality of implementation). We found that P4P had a significant, positive impact on MIBI implementation in HIV service settings, but client-level outcomes were mixed. Future research should examine the effectiveness of the P4P strategy to improve implementation and sustainment of other evidence-based innovations.\n<bold>Trial registration: <\/bold>ClinicalTrials.gov: NCT04687917. Registered 12\/18\/2020.<\/p>","DOI":"10.21203\/rs.3.rs-4614222\/v1","type":"posted-content","created":{"date-parts":[[2024,8,9]],"date-time":"2024-08-09T15:59:51Z","timestamp":1723219191000},"source":"Crossref","is-referenced-by-count":2,"title":["Testing the incremental effectiveness of pay-for-performance to improve implementation of a motivational interviewing brief intervention for substance use disorders in HIV settings: Results of a parallel cluster-randomized type 3 hybrid trial"],"prefix":"10.21203","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7856-9403","authenticated-orcid":false,"given":"Bryan R","family":"Garner","sequence":"first","affiliation":[{"name":"The Ohio State University College of Medicine and Public Health: The Ohio State University College of Medicine"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Stephen J.","family":"Tueller","sequence":"additional","affiliation":[{"name":"RTI International"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael","family":"Bradshaw","sequence":"additional","affiliation":[{"name":"RTI International"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kathryn J.","family":"Speck","sequence":"additional","affiliation":[{"name":"University of Nebraska-Lincoln"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Derek","family":"Satre","sequence":"additional","affiliation":[{"name":"University of California San Francisco"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Carla","family":"Rash","sequence":"additional","affiliation":[{"name":"University of Connecticut Health Center: UConn Health"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tom","family":"Donohoe","sequence":"additional","affiliation":[{"name":"University of California Los Angeles"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jackie","family":"Mungo","sequence":"additional","affiliation":[{"name":"RTI International"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sarah","family":"Philbrick","sequence":"additional","affiliation":[{"name":"RTI International"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Richa","family":"Ruwala","sequence":"additional","affiliation":[{"name":"RTI International"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mathew R.","family":"Roosa","sequence":"additional","affiliation":[{"name":"Roosa Consulting"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mark","family":"Zehner","sequence":"additional","affiliation":[{"name":"University of Wisconsin-Madison"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"James H. Ford","family":"II","sequence":"additional","affiliation":[{"name":"University of Wisconsin-Madison"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","reference":[{"issue":"9","key":"ref1","doi-asserted-by":"publisher","first-page":"844","DOI":"10.1001\/jama.2019.1343","article-title":"Ending the HIV Epidemic: A Plan for the United States","volume":"321","author":"Fauci AS","year":"2019","unstructured":"Fauci AS, Redfield RR, Sigounas G, Weahkee MD, Giroir BP. Ending the HIV Epidemic: A Plan for the United States. JAMA. 2019;321(9):844. 10.1001\/jama.2019.1343.","journal-title":"JAMA"},{"issue":"4","key":"ref2","doi-asserted-by":"publisher","first-page":"1183","DOI":"10.1007\/s10461-021-03473-9","article-title":"The Prevalence and Negative Impacts of Substance Use Disorders among People with HIV in the United States: A Real-Time Delphi Survey of Key Stakeholders","volume":"26","author":"Garner BR","year":"2022","unstructured":"Garner BR, Gotham HJ, Knudsen HK, et al. The Prevalence and Negative Impacts of Substance Use Disorders among People with HIV in the United States: A Real-Time Delphi Survey of Key Stakeholders. Aids Behav. 2022;26(4):1183\u201396. 10.1007\/s10461-021-03473-9.","journal-title":"Aids Behav"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"109150","DOI":"10.1016\/j.drugalcdep.2021.109150","article-title":"Alcohol and other substance use during the COVID-19 pandemic: A systematic review","volume":"229","author":"Roberts A","year":"2021","unstructured":"Roberts A, Rogers J, Mason R, et al. Alcohol and other substance use during the COVID-19 pandemic: A systematic review. Drug Alcohol Depend. 2021;229:109150. 10.1016\/j.drugalcdep.2021.109150.","journal-title":"Drug Alcohol Depend"},{"key":"ref4","doi-asserted-by":"publisher","first-page":"109065","DOI":"10.1016\/j.drugalcdep.2021.109065","article-title":"The early impact of COVID-19 on the incidence, prevalence, and severity of alcohol use and other drugs: A systematic review","volume":"228","author":"Schmidt RA","year":"2021","unstructured":"Schmidt RA, Genois R, Jin J, Vigo D, Rehm J, Rush B. The early impact of COVID-19 on the incidence, prevalence, and severity of alcohol use and other drugs: A systematic review. Drug Alcohol Depend. 2021;228:109065. 10.1016\/j.drugalcdep.2021.109065.","journal-title":"Drug Alcohol Depend"},{"key":"ref5","doi-asserted-by":"publisher","first-page":"106754","DOI":"10.1016\/j.addbeh.2020.106754","article-title":"Substance use and abuse, COVID-19-related distress, and disregard for social distancing: A network analysis","volume":"114","author":"Taylor S","year":"2021","unstructured":"Taylor S, Paluszek MM, Rachor GS, McKay D, Asmundson GJG. Substance use and abuse, COVID-19-related distress, and disregard for social distancing: A network analysis. Addict Behav. 2021;114:106754. 10.1016\/j.addbeh.2020.106754.","journal-title":"Addict Behav"},{"issue":"6","key":"ref6","doi-asserted-by":"publisher","first-page":"e293","DOI":"10.1097\/ADM.0000000000000755","article-title":"Treatment for Anxiety and Substance Use Disorders During the COVID-19 Pandemic: Challenges and Strategies","volume":"14","author":"Satre DD","year":"2020","unstructured":"Satre DD, Iturralde E, Ghadiali M, et al. Treatment for Anxiety and Substance Use Disorders During the COVID-19 Pandemic: Challenges and Strategies. J Addict Med. 2020;14(6):e293\u20136. 10.1097\/ADM.0000000000000755.","journal-title":"J Addict Med"},{"issue":"2","key":"ref7","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1136\/sextrans-2011-050199","article-title":"Costs and efficiency of integrating HIV\/AIDS services with other health services: a systematic review of evidence and experience","volume":"88","author":"Sweeney S","year":"2012","unstructured":"Sweeney S, Obure CD, Maier CB, Greener R, Dehne K, Vassall A. Costs and efficiency of integrating HIV\/AIDS services with other health services: a systematic review of evidence and experience. Sex Transm Infect. 2012;88(2):85\u201399. 10.1136\/sextrans-2011-050199.","journal-title":"Sex Transm Infect"},{"issue":"3","key":"ref8","doi-asserted-by":"publisher","first-page":"214","DOI":"10.1097\/JAN.0000000000000137","article-title":"Brief Intervention, and Referral to Treatment Among People Living With HIV\/AIDS","volume":"27","author":"Savage CL","year":"2016","unstructured":"Savage CL, Sanchez M, Alcohol, Screening SUD. Brief Intervention, and Referral to Treatment Among People Living With HIV\/AIDS. UJAN. 2016;27(3):214\u20137. 10.1097\/JAN.0000000000000137.","journal-title":"UJAN"},{"issue":"1","key":"ref9","doi-asserted-by":"publisher","first-page":"21585","DOI":"10.7448\/IAS.20.1.21585","article-title":"Integrating HIV and substance use services: a systematic review: Haldane V","volume":"20","author":"Haldane V","year":"2017","unstructured":"Haldane V, Cervero-Liceras F, Chuah FL, et al. Integrating HIV and substance use services: a systematic review: Haldane V. J Int AIDS Soc. 2017;20(1):21585. 10.7448\/IAS.20.1.21585.","journal-title":"J Int AIDS Soc"},{"issue":"S2","key":"ref10","doi-asserted-by":"publisher","first-page":"126","DOI":"10.1007\/s10461-017-1886-3","article-title":"Behavioral Interventions Targeting Alcohol Use Among People Living with HIV\/AIDS: A Systematic Review and Meta-Analysis","volume":"21","author":"The MASHR","year":"2017","unstructured":"The MASHR, Team, Scott-Sheldon LAJ, Carey KB, Johnson BT, Carey MP. Behavioral Interventions Targeting Alcohol Use Among People Living with HIV\/AIDS: A Systematic Review and Meta-Analysis. AIDS Behav. 2017;21(S2):126\u201343. 10.1007\/s10461-017-1886-3.","journal-title":"AIDS Behav"},{"issue":"1","key":"ref11","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1007\/s11904-019-00434-9","article-title":"Substance Use Screening in HIV Care Settings: a Review and Critique of the Literature","volume":"16","author":"Hitch AE","year":"2019","unstructured":"Hitch AE, Gause NK, Brown JL. Substance Use Screening in HIV Care Settings: a Review and Critique of the Literature. Curr HIV\/AIDS Rep. 2019;16(1):7\u201316. 10.1007\/s11904-019-00434-9.","journal-title":"Curr HIV\/AIDS Rep"},{"key":"ref12","unstructured":"Substance Abuse and Mental Services Administation. HHS Announces Nearly $44 Million to Strengthen Mental Health and Substance Use Services for Populations at Risk for or Living with HIV\/AIDS. 2022. Accessed March 29, 2023. https:\/\/www.samhsa.gov\/newsroom\/press-announcements\/20220318\/hhs-announces-44-million-mental-health-substance-use-population-hiv-aids#:~:text=Totaling%20%2443.7%20million%20dollars%2C%20the,HHS's%20new%20Overdose%20Prevention%20Strategy."},{"key":"ref13","doi-asserted-by":"publisher","first-page":"263348952094807","DOI":"10.1177\/2633489520948073","article-title":"The implementation and sustainment facilitation strategy improved implementation effectiveness and intervention effectiveness: Results from a cluster-randomized, type 2 hybrid trial","volume":"1","author":"Garner BR","year":"2020","unstructured":"Garner BR, Gotham HJ, Chaple M, et al. The implementation and sustainment facilitation strategy improved implementation effectiveness and intervention effectiveness: Results from a cluster-randomized, type 2 hybrid trial. Implement Res Pract. 2020;1:263348952094807. 10.1177\/2633489520948073.","journal-title":"Implement Res Pract"},{"key":"ref14","doi-asserted-by":"publisher","DOI":"10.1186\/s13722-017-0096-7","author":"Garner BR","year":"2017","unstructured":"Garner BR, Zehner M, Roosa MR et al. Testing the implementation and sustainment facilitation (ISF) strategy as an effective adjunct to the Addiction Technology Transfer Center (ATTC) strategy: study protocol for a cluster randomized trial. Addict Sci Clin Prac. 2017;12. doi:ARTN 32 10.1186\/s13722-017-0096-7."},{"key":"ref15","doi-asserted-by":"publisher","DOI":"10.1186\/s13722-017-0095-8","author":"Garner BR","year":"2017","unstructured":"Garner BR, Gotham HJ, Tueller SJ et al. Testing the effectiveness of a motivational interviewing-based brief intervention for substance use as an adjunct to usual care in community-based AIDS service organizations: study protocol for a multisite randomized controlled trial. Addict Sci Clin Prac. 2017;12. doi:ARTN 31 10.1186\/s13722-017-0095-8."},{"issue":"6","key":"ref16","doi-asserted-by":"publisher","first-page":"1050","DOI":"10.1037\/0022-006X.72.6.1050","article-title":"A Randomized Trial of Methods to Help Clinicians Learn Motivational Interviewing","volume":"72","author":"Miller WR","year":"2004","unstructured":"Miller WR, Yahne CE, Moyers TB, Martinez J, Pirritano M. A Randomized Trial of Methods to Help Clinicians Learn Motivational Interviewing. J Consult Clin Psychol. 2004;72(6):1050\u201362. 10.1037\/0022-006X.72.6.1050.","journal-title":"J Consult Clin Psychol"},{"issue":"4","key":"ref17","doi-asserted-by":"publisher","first-page":"1055","DOI":"10.2307\/259164","article-title":"The Challenge of Innovation Implementation","volume":"21","author":"Klein KJ","year":"1996","unstructured":"Klein KJ, Sorra JS. The Challenge of Innovation Implementation. Acad Manage Rev. 1996;21(4):1055\u201380. 10.2307\/259164.","journal-title":"Acad Manage Rev"},{"issue":"5","key":"ref18","doi-asserted-by":"crossref","first-page":"811","DOI":"10.1037\/0021-9010.86.5.811","article-title":"Implementing computerized technology: an organizational analysis","volume":"86","author":"Klein KJ","year":"2001","unstructured":"Klein KJ, Conn AB, Sorra JS. Implementing computerized technology: an organizational analysis. J Appl Psychol. 2001;86(5):811.","journal-title":"J Appl Psychol"},{"issue":"3","key":"ref19","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1177\/1077558707299887","article-title":"Determinants of Implementation Effectiveness: Adapting a Framework for Complex Innovations","volume":"64","author":"Helfrich CD","year":"2007","unstructured":"Helfrich CD, Weiner BJ, McKinney MM, Minasian L. Determinants of Implementation Effectiveness: Adapting a Framework for Complex Innovations. Med Care Res Rev. 2007;64(3):279\u2013303. 10.1177\/1077558707299887.","journal-title":"Med Care Res Rev"},{"issue":"1","key":"ref20","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1007\/s10488-010-0327-7","article-title":"Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors","volume":"38","author":"Aarons GA","year":"2011","unstructured":"Aarons GA, Hurlburt M, Horwitz SM. Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors. Adm Policy Ment Health. 2011;38(1):4\u201323. 10.1007\/s10488-010-0327-7.","journal-title":"Adm Policy Ment Health"},{"issue":"1","key":"ref21","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s13012-018-0842-6","article-title":"Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework","volume":"14","author":"Moullin JC","year":"2019","unstructured":"Moullin JC, Dickson KS, Stadnick NA, Rabin B, Aarons GA. Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Implement Sci. 2019;14(1):1. 10.1186\/s13012-018-0842-6.","journal-title":"Implement Sci"},{"issue":"6","key":"ref22","doi-asserted-by":"crossref","first-page":"527","DOI":"10.1037\/a0016830","article-title":"Toward a theory of motivational interviewing","volume":"64","author":"Miller WR","year":"2009","unstructured":"Miller WR, Rose GS. Toward a theory of motivational interviewing. Am Psychol. 2009;64(6):527.","journal-title":"Am Psychol"},{"key":"ref23","volume-title":"Motivational Interviewing: Helping People Change","author":"Miller WR","year":"2012","unstructured":"Miller WR, Rollnick S. Motivational Interviewing: Helping People Change. Guilford Press; 2012."},{"key":"ref24","volume-title":"Motivational Interviewing in Groups","author":"Wagner CC","year":"2012","unstructured":"Wagner CC, Ingersoll KS. Motivational Interviewing in Groups. Guilford Press; 2012."},{"key":"ref25","doi-asserted-by":"publisher","DOI":"10.1186\/1748-5908-5-5","author":"Garner BR","year":"2010","unstructured":"Garner BR, Godley SH, Dennis ML, Godley MD, Shepard DS. The Reinforcing Therapist Performance (RTP) experiment: Study protocol for a cluster randomized trial. Implement Sci. 2010;5. doi:Artn 5 10.1186\/1748-5908-5-5."},{"issue":"10","key":"ref26","doi-asserted-by":"publisher","first-page":"938","DOI":"10.1001\/archpediatrics.2012.802","article-title":"Using Pay for Performance to Improve Treatment Implementation for Adolescent Substance Use Disorders Results From a Cluster Randomized Trial","volume":"166","author":"Garner BR","year":"2012","unstructured":"Garner BR, Godley SH, Dennis ML, Hunter BD, Bair CML, Godley MD. Using Pay for Performance to Improve Treatment Implementation for Adolescent Substance Use Disorders Results From a Cluster Randomized Trial. Arch Pediat Adol Med. 2012;166(10):938\u201344. 10.1001\/archpediatrics.2012.802.","journal-title":"Arch Pediat Adol Med"},{"key":"ref27","doi-asserted-by":"publisher","first-page":"ARTN","DOI":"10.1186\/s13012-018-0774-1","article-title":"Pay-for-performance as a cost-effective implementation strategy: results from a cluster randomized trial","volume":"13","author":"Garner BR","year":"2018","unstructured":"Garner BR, Lwin AK, Strickler GK, Hunter BD, Shepard DS. Pay-for-performance as a cost-effective implementation strategy: results from a cluster randomized trial. Implement Sci. 2018;13:ARTN. 92 10.1186\/s13012-018-0774-1.","journal-title":"Implement Sci"},{"issue":"1","key":"ref28","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1186\/s13012-015-0209-1","article-title":"A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project","volume":"10","author":"Powell BJ","year":"2015","unstructured":"Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10(1):21. 10.1186\/s13012-015-0209-1.","journal-title":"Implement Sci"},{"issue":"9","key":"ref29","doi-asserted-by":"crossref","first-page":"936","DOI":"10.1016\/j.jclinepi.2010.12.003","article-title":"Systematic review of stepped wedge cluster randomized trials shows that design is particularly used to evaluate interventions during routine implementation","volume":"64","author":"Mdege ND","year":"2011","unstructured":"Mdege ND, Man MS, Taylor CA, Torgerson DJ. Systematic review of stepped wedge cluster randomized trials shows that design is particularly used to evaluate interventions during routine implementation. J Clin Epidemiol. 2011;64(9):936\u201348.","journal-title":"J Clin Epidemiol"},{"issue":"12","key":"ref30","doi-asserted-by":"crossref","first-page":"1249","DOI":"10.1016\/j.jclinepi.2012.06.004","article-title":"Use of the stepped wedge design cannot be recommended: a critical appraisal and comparison with the classic cluster randomized controlled trial design","volume":"65","author":"Kotz D","year":"2012","unstructured":"Kotz D, Spigt M, Arts IC, Crutzen R, Viechtbauer W. Use of the stepped wedge design cannot be recommended: a critical appraisal and comparison with the classic cluster randomized controlled trial design. J Clin Epidemiol. 2012;65(12):1249\u201352.","journal-title":"J Clin Epidemiol"},{"issue":"12","key":"ref31","doi-asserted-by":"crossref","first-page":"1255","DOI":"10.1016\/j.jclinepi.2012.06.016","article-title":"Researchers should convince policy makers to perform a classic cluster randomized controlled trial instead of a stepped wedge design when an intervention is rolled out","volume":"65","author":"Kotz D","year":"2012","unstructured":"Kotz D, Spigt M, Arts IC, Crutzen R, Viechtbauer W. Researchers should convince policy makers to perform a classic cluster randomized controlled trial instead of a stepped wedge design when an intervention is rolled out. J Clin Epidemiol. 2012;65(12):1255\u20136.","journal-title":"J Clin Epidemiol"},{"issue":"9","key":"ref32","doi-asserted-by":"crossref","first-page":"1059","DOI":"10.1016\/j.jclinepi.2013.05.004","article-title":"The stepped wedge design does not inherently have more power than a cluster randomized controlled trial","volume":"66","author":"Kotz D","year":"2013","unstructured":"Kotz D, Spigt M, Arts IC, Crutzen R, Viechtbauer W. The stepped wedge design does not inherently have more power than a cluster randomized controlled trial. J Clin Epidemiol. 2013;66(9):1059\u201360.","journal-title":"J Clin Epidemiol"},{"issue":"1","key":"ref33","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1007\/s10488-008-0197-4","article-title":"Implementation Research in Mental Health Services: an Emerging Science with Conceptual, Methodological, and Training challenges","volume":"36","author":"Proctor EK","year":"2009","unstructured":"Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation Research in Mental Health Services: an Emerging Science with Conceptual, Methodological, and Training challenges. Adm Policy Ment Health. 2009;36(1):24\u201334. 10.1007\/s10488-008-0197-4.","journal-title":"Adm Policy Ment Health"},{"issue":"3","key":"ref34","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1097\/MLR.0b013e3182408812","article-title":"Effectiveness-implementation Hybrid Designs: Combining Elements of Clinical Effectiveness and Implementation Research to Enhance Public Health Impact","volume":"50","author":"Curran GM","year":"2012","unstructured":"Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation Hybrid Designs: Combining Elements of Clinical Effectiveness and Implementation Research to Enhance Public Health Impact. Med Care. 2012;50(3):217\u201326. 10.1097\/MLR.0b013e3182408812.","journal-title":"Med Care"},{"issue":"1","key":"ref35","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1186\/s13012-015-0240-2","article-title":"Implementation science: a reappraisal of our journal mission and scope","volume":"10","author":"Foy R","year":"2015","unstructured":"Foy R, Sales A, Wensing M, et al. Implementation science: a reappraisal of our journal mission and scope. Implement Sci. 2015;10(1):51. 10.1186\/s13012-015-0240-2.","journal-title":"Implement Sci"},{"issue":"1","key":"ref36","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1186\/s13012-020-01041-8","article-title":"The Implementation Research Logic Model: a method for planning, executing, reporting, and synthesizing implementation projects","volume":"15","author":"Smith JD","year":"2020","unstructured":"Smith JD, Li DH, Rafferty MR. The Implementation Research Logic Model: a method for planning, executing, reporting, and synthesizing implementation projects. Implement Sci. 2020;15(1):84. 10.1186\/s13012-020-01041-8.","journal-title":"Implement Sci"},{"key":"ref37","doi-asserted-by":"publisher","first-page":"1053496","DOI":"10.3389\/frhs.2022.1053496","article-title":"Reflections on 10 years of effectiveness-implementation hybrid studies","volume":"2","author":"Curran GM","year":"2022","unstructured":"Curran GM, Landes SJ, McBain SA, et al. Reflections on 10 years of effectiveness-implementation hybrid studies. FrontHealth Serv. 2022;2:1053496. 10.3389\/frhs.2022.1053496.","journal-title":"FrontHealth Serv"},{"issue":"2","key":"ref38","doi-asserted-by":"publisher","first-page":"330","DOI":"10.1007\/s10865-018-9988-6","article-title":"The association between symptoms of generalized anxiety disorder and appointment adherence, overnight hospitalization, and emergency department\/urgent care visits among adults living with HIV enrolled in care","volume":"42","author":"Mannes ZL","year":"2019","unstructured":"Mannes ZL, Hearn LE, Zhou Z, Janelle JW, Cook RL, Ennis N. The association between symptoms of generalized anxiety disorder and appointment adherence, overnight hospitalization, and emergency department\/urgent care visits among adults living with HIV enrolled in care. J Behav Med. 2019;42(2):330\u201341. 10.1007\/s10865-018-9988-6.","journal-title":"J Behav Med"},{"key":"ref39","doi-asserted-by":"publisher","first-page":"168","DOI":"10.1016\/j.jad.2017.02.016","article-title":"Patterns of marijuana use among psychiatry patients with depression and its impact on recovery","volume":"213","author":"Bahorik AL","year":"2017","unstructured":"Bahorik AL, Leibowitz A, Sterling SA, Travis A, Weisner C, Satre DD. Patterns of marijuana use among psychiatry patients with depression and its impact on recovery. J Affect Disord. 2017;213:168\u201371. 10.1016\/j.jad.2017.02.016.","journal-title":"J Affect Disord"},{"key":"ref40","doi-asserted-by":"publisher","first-page":"110990","DOI":"10.1016\/j.drugalcdep.2023.110990","article-title":"Integrated behavioral treatments for comorbid anxiety and substance use disorders: A model for understanding integrated treatment approaches and meta-analysis to evaluate their efficacy","volume":"253","author":"Wolitzky-Taylor K","year":"2023","unstructured":"Wolitzky-Taylor K. Integrated behavioral treatments for comorbid anxiety and substance use disorders: A model for understanding integrated treatment approaches and meta-analysis to evaluate their efficacy. Drug Alcohol Depend. 2023;253:110990. 10.1016\/j.drugalcdep.2023.110990.","journal-title":"Drug Alcohol Depend"},{"key":"ref41","doi-asserted-by":"publisher","first-page":"263348952210892","DOI":"10.1177\/26334895221089266","article-title":"The implementation & sustainment facilitation (ISF) strategy: Cost and cost-effectiveness results from a 39-site cluster randomized trial integrating substance use services in community-based HIV service organizations","volume":"3","author":"Hinde JM","year":"2022","unstructured":"Hinde JM, Garner BR, Watson CJ, Ramanan R, Ball EL, Tueller SJ. The implementation & sustainment facilitation (ISF) strategy: Cost and cost-effectiveness results from a 39-site cluster randomized trial integrating substance use services in community-based HIV service organizations. Implement Res Pract. 2022;3:263348952210892. 10.1177\/26334895221089266.","journal-title":"Implement Res Pract"},{"issue":"1","key":"ref42","doi-asserted-by":"publisher","first-page":"46","DOI":"10.1186\/1748-5908-9-46","article-title":"Context matters: measuring implementation climate among individuals and groups","volume":"9","author":"Jacobs SR","year":"2014","unstructured":"Jacobs SR, Weiner BJ, Bunger AC. Context matters: measuring implementation climate among individuals and groups. Implement Sci. 2014;9(1):46. 10.1186\/1748-5908-9-46.","journal-title":"Implement Sci"},{"key":"ref43","doi-asserted-by":"publisher","first-page":"S206","DOI":"10.1097\/Qai.0000000000002981","article-title":"The Setting-Intervention Fit of Nine Evidence-Based Interventions for Substance Use Disorders Within HIV Service Organizations Across the United States: Results of a National Stakeholder-Engaged Real-Time Delphi Study","volume":"90","author":"Garner BR","year":"2022","unstructured":"Garner BR, Gotham HJ, Knudsen HK, et al. The Setting-Intervention Fit of Nine Evidence-Based Interventions for Substance Use Disorders Within HIV Service Organizations Across the United States: Results of a National Stakeholder-Engaged Real-Time Delphi Study. Jaids-J Acq Imm Def. 2022;90:S206\u201314. 10.1097\/Qai.0000000000002981.","journal-title":"Jaids-J Acq Imm Def"},{"issue":"1","key":"ref44","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1186\/1748-5908-9-7","article-title":"Organizational readiness for implementing change: a psychometric assessment of a new measure","volume":"9","author":"Shea CM","year":"2014","unstructured":"Shea CM, Jacobs SR, Esserman DA, Bruce K, Weiner BJ. Organizational readiness for implementing change: a psychometric assessment of a new measure. Implement Sci. 2014;9(1):7. 10.1186\/1748-5908-9-7.","journal-title":"Implement Sci"},{"key":"ref45","author":"Association D","year":"2013","unstructured":"American Psychiatric Association, Association D. AP. Diagnostic and Statistical Manual of Mental Disorders: DSM-5. Vol 5. American psychiatric association Washington, DC; 2013."},{"issue":"3","key":"ref46","doi-asserted-by":"publisher","first-page":"324","DOI":"10.1007\/s11414-013-9381-8","article-title":"Parallel Processes: Using Motivational Interviewing as an Implementation Coaching Strategy","volume":"41","author":"Hettema JE","year":"2014","unstructured":"Hettema JE, Ernst D, Williams JR, Miller KJ. Parallel Processes: Using Motivational Interviewing as an Implementation Coaching Strategy. J Behav Health Serv Res. 2014;41(3):324\u201336. 10.1007\/s11414-013-9381-8.","journal-title":"J Behav Health Serv Res"},{"issue":"1","key":"ref47","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1748-5908-5-75","article-title":"Employing external facilitation to implement cognitive behavioral therapy in VA clinics: a pilot study","volume":"5","author":"Kauth MR","year":"2010","unstructured":"Kauth MR, Sullivan G, Blevins D, et al. Employing external facilitation to implement cognitive behavioral therapy in VA clinics: a pilot study. Implement Sci. 2010;5(1):1\u201311.","journal-title":"Implement Sci"},{"issue":"2","key":"ref48","doi-asserted-by":"crossref","first-page":"318","DOI":"10.1037\/pst0000221","article-title":"Design feasibility of an automated, machine-learning based feedback system for motivational interviewing","volume":"56","author":"Imel ZE","year":"2019","unstructured":"Imel ZE, Pace BT, Soma CS, et al. Design feasibility of an automated, machine-learning based feedback system for motivational interviewing. Psychotherapy. 2019;56(2):318.","journal-title":"Psychotherapy"},{"issue":"3","key":"ref49","doi-asserted-by":"crossref","first-page":"322","DOI":"10.1016\/j.ajog.2017.12.005","article-title":"A randomized controlled trial of screening and brief interventions for substance misuse in reproductive health","volume":"218","author":"Martino S","year":"2018","unstructured":"Martino S, Ondersma SJ, Forray A, et al. A randomized controlled trial of screening and brief interventions for substance misuse in reproductive health. Am J Obstet Gynecol. 2018;218(3):322. e1-322. e12.","journal-title":"Am J Obstet Gynecol"},{"key":"ref50","doi-asserted-by":"publisher","first-page":"64","DOI":"10.3389\/fpubh.2019.00064","article-title":"RE-AIM Planning and Evaluation Framework: Adapting to New Science and Practice With a 20-Year Review","volume":"7","author":"Glasgow RE","year":"2019","unstructured":"Glasgow RE, Harden SM, Gaglio B, et al. RE-AIM Planning and Evaluation Framework: Adapting to New Science and Practice With a 20-Year Review. Front Public Health. 2019;7:64. 10.3389\/fpubh.2019.00064.","journal-title":"Front Public Health"},{"issue":"2","key":"ref51","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1007\/s10488-010-0319-7","article-title":"Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda","volume":"38","author":"Proctor E","year":"2011","unstructured":"Proctor E, Silmere H, Raghavan R, et al. Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda. Adm Policy Ment Health. 2011;38(2):65\u201376. 10.1007\/s10488-010-0319-7.","journal-title":"Adm Policy Ment Health"},{"issue":"1","key":"ref52","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/S0272-7358(97)00043-3","article-title":"Program integrity in primary and early secondary prevention: are implementation effects out of control?","volume":"18","author":"Dane AV","year":"1998","unstructured":"Dane AV, Schneider BH. Program integrity in primary and early secondary prevention: are implementation effects out of control? Clin Psychol Rev. 1998;18(1):23\u201345.","journal-title":"Clin Psychol Rev"},{"issue":"10","key":"ref53","doi-asserted-by":"crossref","first-page":"1092","DOI":"10.1001\/archinte.166.10.1092","article-title":"A brief measure for assessing generalized anxiety disorder: the GAD-7","volume":"166","author":"Spitzer RL","year":"2006","unstructured":"Spitzer RL, Kroenke K, Williams JB, L\u00f6we B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092\u20137.","journal-title":"Arch Intern Med"},{"key":"ref54","unstructured":"SAS version 9.4. (2023). SAS Institute Inc. Cary, NC, USA."},{"issue":"427","key":"ref55","doi-asserted-by":"crossref","first-page":"846","DOI":"10.1080\/01621459.1994.10476818","article-title":"Estimation of regression coefficients when some regressors are not always observed","volume":"89","author":"Robins JM","year":"1994","unstructured":"Robins JM, Rotnitzky A, Zhao LP. Estimation of regression coefficients when some regressors are not always observed. J Am Stat Assoc. 1994;89(427):846\u201366.","journal-title":"J Am Stat Assoc"},{"issue":"4","key":"ref56","doi-asserted-by":"crossref","first-page":"962","DOI":"10.1111\/j.1541-0420.2005.00377.x","article-title":"Doubly robust estimation in missing data and causal inference models","volume":"61","author":"Bang H","year":"2005","unstructured":"Bang H, Robins JM. Doubly robust estimation in missing data and causal inference models. Biometrics. 2005;61(4):962\u201373.","journal-title":"Biometrics"},{"issue":"7","key":"ref57","doi-asserted-by":"publisher","first-page":"761","DOI":"10.1093\/aje\/kwq439","article-title":"Doubly Robust Estimation of Causal Effects","volume":"173","author":"Funk MJ","year":"2011","unstructured":"Funk MJ, Westreich D, Wiesen C, St\u00fcrmer T, Brookhart MA, Davidian M. Doubly Robust Estimation of Causal Effects. Am J Epidemiol. 2011;173(7):761\u20137. 10.1093\/aje\/kwq439.","journal-title":"Am J Epidemiol"},{"issue":"511","key":"ref58","doi-asserted-by":"crossref","first-page":"1024","DOI":"10.1080\/01621459.2014.958155","article-title":"Bias-reduced doubly robust estimation","volume":"110","author":"Vermeulen K","year":"2015","unstructured":"Vermeulen K, Vansteelandt S. Bias-reduced doubly robust estimation. J Am Stat Assoc. 2015;110(511):1024\u201336.","journal-title":"J Am Stat Assoc"},{"issue":"1","key":"ref59","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1186\/s13722-019-0150-8","article-title":"HIV care provider perceptions and approaches to managing unhealthy alcohol use in primary HIV care settings: a qualitative study","volume":"14","author":"Chichetto NE","year":"2019","unstructured":"Chichetto NE, Mannes ZL, Allen MK, Cook RL, Ennis N. HIV care provider perceptions and approaches to managing unhealthy alcohol use in primary HIV care settings: a qualitative study. Addict Sci Clin Pract. 2019;14(1):21. 10.1186\/s13722-019-0150-8.","journal-title":"Addict Sci Clin Pract"},{"key":"ref60","first-page":"1","article-title":"Serving the co-morbid mental health and substance use needs of people with HIV","volume":"2021","author":"Conway FN","unstructured":"Conway FN, Rountree MA, Jones KV. Serving the co-morbid mental health and substance use needs of people with HIV. Community Mental Health J Published online 2021:1\u201312.","journal-title":"Community Mental Health J Published online"},{"issue":"8","key":"ref61","doi-asserted-by":"publisher","first-page":"ofac365","DOI":"10.1093\/ofid\/ofac365","article-title":"Interventions for Integrating Behavioral Health Services Into HIV Clinical Care: A Narrative Review","volume":"9","author":"Goldhammer H","year":"2022","unstructured":"Goldhammer H, Marc LG, Chavis NS, et al. Interventions for Integrating Behavioral Health Services Into HIV Clinical Care: A Narrative Review. Open Forum Infect Dis. 2022;9(8):ofac365. 10.1093\/ofid\/ofac365.","journal-title":"Open Forum Infect Dis"},{"issue":"1","key":"ref62","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1186\/s13011-017-0104-7","article-title":"Strategies to facilitate integrated care for people with alcohol and other drug problems: a systematic review","volume":"12","author":"Savic M","year":"2017","unstructured":"Savic M, Best D, Manning V, Lubman DI. Strategies to facilitate integrated care for people with alcohol and other drug problems: a systematic review. Subst Abuse Treat Prev Policy. 2017;12(1):19. 10.1186\/s13011-017-0104-7.","journal-title":"Subst Abuse Treat Prev Policy"},{"issue":"7748","key":"ref63","doi-asserted-by":"crossref","first-page":"305","DOI":"10.1038\/d41586-019-00857-9","article-title":"Scientists rise up against statistical significance","volume":"567","author":"Amrhein V","year":"2019","unstructured":"Amrhein V, Greenland S, McShane B. Scientists rise up against statistical significance. Nature. 2019;567(7748):305\u20137.","journal-title":"Nature"},{"issue":"12","key":"ref64","doi-asserted-by":"crossref","first-page":"1219","DOI":"10.1001\/jamapsychiatry.2019.1965","article-title":"Is it time to ban the P value?","volume":"76","author":"Kraemer HC","year":"2019","unstructured":"Kraemer HC. Is it time to ban the P value? JAMA psychiatry. 2019;76(12):1219\u201320.","journal-title":"JAMA psychiatry"},{"key":"ref65","first-page":"1","volume-title":"Moving to a World beyond P\u2009<\u20090.05","author":"Wasserstein RL","year":"2019","unstructured":"Wasserstein RL, Schirm AL, Lazar NA. Moving to a World beyond P\u2009<\u20090.05. Volume 73. Taylor & Francis; 2019. pp. 1\u201319."},{"key":"ref66","author":"Kim JH","unstructured":"Kim JH, Gerhart B, Fang M. Do Financial Incentives Help or Harm Performance in Interesting Tasks?."},{"key":"ref67","article-title":"Cognitive evaluation theory and some comments on the Calder and Staw critique","author":"Deci EL","unstructured":"Deci EL, Cascio WF, Krusell J. Cognitive evaluation theory and some comments on the Calder and Staw critique. Published online 1975."},{"issue":"6","key":"ref68","doi-asserted-by":"crossref","first-page":"627","DOI":"10.1037\/0033-2909.125.6.627","article-title":"A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation","volume":"125","author":"Deci EL","year":"1999","unstructured":"Deci EL, Koestner R, Ryan RM. A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychol Bull. 1999;125(6):627.","journal-title":"Psychol Bull"},{"issue":"2","key":"ref69","doi-asserted-by":"crossref","first-page":"147","DOI":"10.2224\/sbp.1988.16.2.147","article-title":"Cognitive evaluation theory: A meta-analytic review of the literature","volume":"16","author":"Rummel A","year":"1988","unstructured":"Rummel A, Feinberg R. Cognitive evaluation theory: A meta-analytic review of the literature. Social Behav Personality: Int J. 1988;16(2):147\u201364.","journal-title":"Social Behav Personality: Int J"},{"issue":"2","key":"ref70","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1016\/0092-6566(85)90023-6","article-title":"The general causality orientations scale: Self-determination in personality","volume":"19","author":"Deci EL","year":"1985","unstructured":"Deci EL, Ryan RM. The general causality orientations scale: Self-determination in personality. J Res Pers. 1985;19(2):109\u201334.","journal-title":"J Res Pers"},{"key":"ref71","author":"Deci EL","unstructured":"Deci EL, Ryan RM. Self-determination theory. Published online 2012."},{"key":"ref72","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1146\/annurev-orgpsych-032516-113108","article-title":"Self-determination theory in work organizations: The state of a science","volume":"4","author":"Deci EL","year":"2017","unstructured":"Deci EL, Olafsen AH, Ryan RM. Self-determination theory in work organizations: The state of a science. Annual Rev organizational Psychol organizational Behav. 2017;4:19\u201343.","journal-title":"Annual Rev organizational Psychol organizational Behav"},{"key":"ref73","author":"Ryan RM","unstructured":"Ryan RM, Deci EL. Self-determination theory. Basic psychological needs in motivation, development, and wellness. Published online 2017."},{"issue":"2","key":"ref74","doi-asserted-by":"crossref","first-page":"565","DOI":"10.2466\/21.02.PR0.115c23z1","article-title":"Effect of client-therapist gender match on the therapeutic relationship: An exploratory analysis","volume":"115","author":"Bhati KS","year":"2014","unstructured":"Bhati KS. Effect of client-therapist gender match on the therapeutic relationship: An exploratory analysis. Psychol Rep. 2014;115(2):565\u201383.","journal-title":"Psychol Rep"}],"container-title":[],"original-title":[],"link":[{"URL":"https:\/\/www.researchsquare.com\/article\/rs-4614222\/v1","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.researchsquare.com\/article\/rs-4614222\/v1.html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,9]],"date-time":"2024-08-09T16:00:24Z","timestamp":1723219224000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.researchsquare.com\/article\/rs-4614222\/v1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,9]]},"references-count":74,"URL":"https:\/\/doi.org\/10.21203\/rs.3.rs-4614222\/v1","relation":{},"subject":[],"published":{"date-parts":[[2024,8,9]]},"subtype":"preprint"}}