{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"institution":[{"name":"Research Square"}],"indexed":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T14:59:17Z","timestamp":1791385157386,"version":"4.3.3"},"posted":{"date-parts":[[2023,12,20]]},"group-title":"In Review","reference-count":78,"publisher":"Springer Science and Business Media LLC","license":[{"start":{"date-parts":[[2023,12,20]],"date-time":"2023-12-20T00:00:00Z","timestamp":1703030400000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"accepted":{"date-parts":[[2023,11,30]]},"abstract":"<title>Abstract<\/title>\n        <p><bold>Background:<\/bold> Substance use disorders (SUDs) have a serious adverse impact on people living with HIV. Previously, using a 39-site dual-randomized type 2 hybrid trial design, findings from the Substance Abuse Treatment to HIV Care Project supported the Implementation and Sustainment Facilitation (ISF) strategy to improve implementation and effectiveness of a motivational interviewing brief intervention (MIBI) for SUD within HIV service settings across the United States (US). Building on this trial, this cluster-randomized type 3 hybrid trial aimed to test the incremental effectiveness of a pay-for-performance (P4P), a form of the \u201calter incentive\/allowance structures\u201d strategy.\n<bold>Methods:<\/bold> Twenty-six HIV service organizations, their staff participants (N=87), and their client participants (N=341) were cluster-randomized to one of two implementation conditions. The control condition included staff-focused training, feedback, and consultation (TFC) and team-focused implementation and sustainment (ISF). The experimental condition included TFC+ISF as well as P4P (TFC+ISF+P4P). P4P used financial incentives to reward MIBI implementation (US$10 per MIBI delivered) and MIBI implementation at or above a pre-defined level of quality (US$10 per demonstration). In addition to these outcomes, past 4-week changes\/reductions in client participant\u2019s days of primary substance use and anxiety symptoms were examined.\n<bold>Results:<\/bold> The addition of P4P had a large and significant effect on the number of MIBIs implemented (<italic>d<\/italic>=1.30, p&lt;.05) and reduction in anxiety (<italic>d=<\/italic>-1.54), but there was no impact on days of substance use. P4P had large effects on MIBI quality (<italic>d<\/italic>=1.24) and MIBI implementation effectiveness (<italic>d<\/italic>=1.28), but these were not significant (<italic>p<\/italic>&lt;.10).\n<bold>Conclusions:<\/bold> P4P is a form of the \u201calter incentive\/allowance structures\u201d strategy Its function is to reward the implementation of a clinical innovation. Rewarding implementation is consistent with the theory of implementation effectiveness, which suggests implementation climate (i.e., the extent to which implementation is expected, supported, and rewarded) is a key antecedent of implementation effectiveness (i.e., the consistency and quality of implementation). We found that P4P had a significant, positive impact on MIBI implementation in HIV service settings, but client-level outcomes were mixed. Future research should examine the cost-effectiveness of this strategy, as well as to examine the effectiveness of P4P to improve the implementation of other evidence-based innovations.\n<bold>Trial registration: <\/bold>ClinicalTrials.gov: NCT04687917. Registered 12\/18\/2020.<\/p>","DOI":"10.21203\/rs.3.rs-3685134\/v1","type":"posted-content","created":{"date-parts":[[2023,12,20]],"date-time":"2023-12-20T17:45:38Z","timestamp":1703094338000},"source":"Crossref","is-referenced-by-count":1,"title":["Testing the incremental effectiveness of pay-for-performance to improve implementation of a motivational interviewing brief intervention for substance use disorders in HIV settings: Results of a cluster-randomized type 3 hybrid trial"],"prefix":"10.21203","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7856-9403","authenticated-orcid":false,"given":"Bryan R","family":"Garner","sequence":"first","affiliation":[{"name":"The Ohio State University College of Medicine and Public Health: The Ohio State University College of Medicine"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stephen","family":"Tueller","sequence":"additional","affiliation":[{"name":"RTI International"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Bradshaw","sequence":"additional","affiliation":[{"name":"RTI International"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kathryn","family":"Speck","sequence":"additional","affiliation":[{"name":"UNL: University of Nebraska-Lincoln"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Derek","family":"Satre","sequence":"additional","affiliation":[{"name":"University of California San Francisco"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carla","family":"Rash","sequence":"additional","affiliation":[{"name":"UConn Health"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tom","family":"Donohoe","sequence":"additional","affiliation":[{"name":"University of California Los Angeles"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jackie","family":"Mungo","sequence":"additional","affiliation":[{"name":"RTI International"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sarah","family":"Philbrick","sequence":"additional","affiliation":[{"name":"RTI International"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Richa","family":"Ruwala","sequence":"additional","affiliation":[{"name":"RTI International"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mathew","family":"Roosa","sequence":"additional","affiliation":[{"name":"Roosa Consulting"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mark","family":"Zehner","sequence":"additional","affiliation":[{"name":"University of Wisconsin Madison"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"James Ford","family":"II","sequence":"additional","affiliation":[{"name":"University of Wisconsin Madison"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","reference":[{"issue":"9","key":"ref1","doi-asserted-by":"publisher","first-page":"844","DOI":"10.1001\/jama.2019.1343","article-title":"Ending the HIV Epidemic: A Plan for the United States","volume":"321","author":"Fauci AS","year":"2019","unstructured":"Fauci AS, Redfield RR, Sigounas G, Weahkee MD, Giroir BP. Ending the HIV Epidemic: A Plan for the United States. JAMA. 2019;321(9):844. 10.1001\/jama.2019.1343.","journal-title":"JAMA"},{"issue":"4","key":"ref2","doi-asserted-by":"publisher","first-page":"1183","DOI":"10.1007\/s10461-021-03473-9","article-title":"The Prevalence and Negative Impacts of Substance Use Disorders among People with HIV in the United States: A Real-Time Delphi Survey of Key Stakeholders","volume":"26","author":"Garner BR","year":"2022","unstructured":"Garner BR, Gotham HJ, Knudsen HK, et al. The Prevalence and Negative Impacts of Substance Use Disorders among People with HIV in the United States: A Real-Time Delphi Survey of Key Stakeholders. Aids Behav. 2022;26(4):1183\u201396. 10.1007\/s10461-021-03473-9.","journal-title":"Aids Behav"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"109150","DOI":"10.1016\/j.drugalcdep.2021.109150","article-title":"Alcohol and other substance use during the COVID-19 pandemic: A systematic review","volume":"229","author":"Roberts A","year":"2021","unstructured":"Roberts A, Rogers J, Mason R, et al. Alcohol and other substance use during the COVID-19 pandemic: A systematic review. Drug Alcohol Depend. 2021;229:109150. 10.1016\/j.drugalcdep.2021.109150.","journal-title":"Drug Alcohol Depend"},{"key":"ref4","doi-asserted-by":"publisher","first-page":"109065","DOI":"10.1016\/j.drugalcdep.2021.109065","article-title":"The early impact of COVID-19 on the incidence, prevalence, and severity of alcohol use and other drugs: A systematic review","volume":"228","author":"Schmidt RA","year":"2021","unstructured":"Schmidt RA, Genois R, Jin J, Vigo D, Rehm J, Rush B. The early impact of COVID-19 on the incidence, prevalence, and severity of alcohol use and other drugs: A systematic review. Drug Alcohol Depend. 2021;228:109065. 10.1016\/j.drugalcdep.2021.109065.","journal-title":"Drug Alcohol Depend"},{"key":"ref5","doi-asserted-by":"publisher","first-page":"106754","DOI":"10.1016\/j.addbeh.2020.106754","article-title":"Substance use and abuse, COVID-19-related distress, and disregard for social distancing: A network analysis","volume":"114","author":"Taylor S","year":"2021","unstructured":"Taylor S, Paluszek MM, Rachor GS, McKay D, Asmundson GJG. Substance use and abuse, COVID-19-related distress, and disregard for social distancing: A network analysis. Addict Behav. 2021;114:106754. 10.1016\/j.addbeh.2020.106754.","journal-title":"Addict Behav"},{"issue":"6","key":"ref6","doi-asserted-by":"publisher","first-page":"e293","DOI":"10.1097\/ADM.0000000000000755","article-title":"Treatment for Anxiety and Substance Use Disorders During the COVID-19 Pandemic: Challenges and Strategies","volume":"14","author":"Satre DD","year":"2020","unstructured":"Satre DD, Iturralde E, Ghadiali M, et al. Treatment for Anxiety and Substance Use Disorders During the COVID-19 Pandemic: Challenges and Strategies. J Addict Med. 2020;14(6):e293\u20136. 10.1097\/ADM.0000000000000755.","journal-title":"J Addict Med"},{"issue":"2","key":"ref7","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1136\/sextrans-2011-050199","article-title":"Costs and efficiency of integrating HIV\/AIDS services with other health services: a systematic review of evidence and experience","volume":"88","author":"Sweeney S","year":"2012","unstructured":"Sweeney S, Obure CD, Maier CB, Greener R, Dehne K, Vassall A. Costs and efficiency of integrating HIV\/AIDS services with other health services: a systematic review of evidence and experience. Sex Transm Infect. 2012;88(2):85\u201399. 10.1136\/sextrans-2011-050199.","journal-title":"Sex Transm Infect"},{"issue":"3","key":"ref8","doi-asserted-by":"publisher","first-page":"214","DOI":"10.1097\/JAN.0000000000000137","article-title":"Brief Intervention, and Referral to Treatment Among People Living With HIV\/AIDS","volume":"27","author":"Savage CL","year":"2016","unstructured":"Savage CL, Sanchez M, Alcohol, Screening SUD. Brief Intervention, and Referral to Treatment Among People Living With HIV\/AIDS. UJAN. 2016;27(3):214\u20137. 10.1097\/JAN.0000000000000137.","journal-title":"UJAN"},{"issue":"1","key":"ref9","doi-asserted-by":"publisher","first-page":"21585","DOI":"10.7448\/IAS.20.1.21585","article-title":"Integrating HIV and substance use services: a systematic review: Haldane V","volume":"20","author":"Haldane V","year":"2017","unstructured":"Haldane V, Cervero-Liceras F, Chuah FL, et al. Integrating HIV and substance use services: a systematic review: Haldane V. J Int AIDS Soc. 2017;20(1):21585. 10.7448\/IAS.20.1.21585.","journal-title":"J Int AIDS Soc"},{"issue":"S2","key":"ref10","doi-asserted-by":"publisher","first-page":"126","DOI":"10.1007\/s10461-017-1886-3","article-title":"Behavioral Interventions Targeting Alcohol Use Among People Living with HIV\/AIDS: A Systematic Review and Meta-Analysis","volume":"21","author":"The MASHR","year":"2017","unstructured":"The MASHR, Team, Scott-Sheldon LAJ, Carey KB, Johnson BT, Carey MP. Behavioral Interventions Targeting Alcohol Use Among People Living with HIV\/AIDS: A Systematic Review and Meta-Analysis. AIDS Behav. 2017;21(S2):126\u201343. 10.1007\/s10461-017-1886-3.","journal-title":"AIDS Behav"},{"issue":"1","key":"ref11","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1007\/s11904-019-00434-9","article-title":"Substance Use Screening in HIV Care Settings: a Review and Critique of the Literature","volume":"16","author":"Hitch AE","year":"2019","unstructured":"Hitch AE, Gause NK, Brown JL. Substance Use Screening in HIV Care Settings: a Review and Critique of the Literature. Curr HIV\/AIDS Rep. 2019;16(1):7\u201316. 10.1007\/s11904-019-00434-9.","journal-title":"Curr HIV\/AIDS Rep"},{"key":"ref12","unstructured":"Substance Abuse and Mental Services Administation. HHS Announces Nearly $44 Million to Strengthen Mental Health and Substance Use Services for Populations at Risk for or Living with HIV\/AIDS. ; 2022. Accessed March 29, 2023. https:\/\/www.samhsa.gov\/newsroom\/press-announcements\/20220318\/hhs-announces-44-million-mental-health-substance-use-population-hiv-aids#:~:text=Totaling%20%2443.7%20million%20dollars%2C%20the,HHS's%20new%20Overdose%20Prevention%20Strategy."},{"key":"ref13","doi-asserted-by":"publisher","first-page":"263348952094807","DOI":"10.1177\/2633489520948073","article-title":"The implementation and sustainment facilitation strategy improved implementation effectiveness and intervention effectiveness: Results from a cluster-randomized, type 2 hybrid trial","volume":"1","author":"Garner BR","year":"2020","unstructured":"Garner BR, Gotham HJ, Chaple M, et al. The implementation and sustainment facilitation strategy improved implementation effectiveness and intervention effectiveness: Results from a cluster-randomized, type 2 hybrid trial. Implement Res Pract. 2020;1:263348952094807. 10.1177\/2633489520948073.","journal-title":"Implement Res Pract"},{"key":"ref14","doi-asserted-by":"publisher","DOI":"10.1186\/s13722-017-0096-7","author":"Garner BR","year":"2017","unstructured":"Garner BR, Zehner M, Roosa MR et al. Testing the implementation and sustainment facilitation (ISF) strategy as an effective adjunct to the Addiction Technology Transfer Center (ATTC) strategy: study protocol for a cluster randomized trial. Addict Sci Clin Prac. 2017;12. doi:ARTN 32 10.1186\/s13722-017-0096-7."},{"key":"ref15","doi-asserted-by":"publisher","DOI":"10.1186\/s13722-017-0095-8","author":"Garner BR","year":"2017","unstructured":"Garner BR, Gotham HJ, Tueller SJ et al. Testing the effectiveness of a motivational interviewing-based brief intervention for substance use as an adjunct to usual care in community-based AIDS service organizations: study protocol for a multisite randomized controlled trial. Addict Sci Clin Prac. 2017;12. doi:ARTN 31 10.1186\/s13722-017-0095-8."},{"issue":"6","key":"ref16","doi-asserted-by":"publisher","first-page":"1050","DOI":"10.1037\/0022-006X.72.6.1050","article-title":"A Randomized Trial of Methods to Help Clinicians Learn Motivational Interviewing","volume":"72","author":"Miller WR","year":"2004","unstructured":"Miller WR, Yahne CE, Moyers TB, Martinez J, Pirritano M. A Randomized Trial of Methods to Help Clinicians Learn Motivational Interviewing. J Consult Clin Psychol. 2004;72(6):1050\u201362. 10.1037\/0022-006X.72.6.1050.","journal-title":"J Consult Clin Psychol"},{"issue":"4","key":"ref17","doi-asserted-by":"publisher","first-page":"1055","DOI":"10.2307\/259164","article-title":"The Challenge of Innovation Implementation","volume":"21","author":"Klein KJ","year":"1996","unstructured":"Klein KJ, Sorra JS. The Challenge of Innovation Implementation. Acad Manage Rev. 1996;21(4):1055\u201380. 10.2307\/259164.","journal-title":"Acad Manage Rev"},{"issue":"5","key":"ref18","doi-asserted-by":"crossref","first-page":"811","DOI":"10.1037\/0021-9010.86.5.811","article-title":"Implementing computerized technology: an organizational analysis","volume":"86","author":"Klein KJ","year":"2001","unstructured":"Klein KJ, Conn AB, Sorra JS. Implementing computerized technology: an organizational analysis. J Appl Psychol. 2001;86(5):811.","journal-title":"J Appl Psychol"},{"issue":"3","key":"ref19","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1177\/1077558707299887","article-title":"Determinants of Implementation Effectiveness: Adapting a Framework for Complex Innovations","volume":"64","author":"Helfrich CD","year":"2007","unstructured":"Helfrich CD, Weiner BJ, McKinney MM, Minasian L. Determinants of Implementation Effectiveness: Adapting a Framework for Complex Innovations. Med Care Res Rev. 2007;64(3):279\u2013303. 10.1177\/1077558707299887.","journal-title":"Med Care Res Rev"},{"issue":"1","key":"ref20","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1007\/s10488-010-0327-7","article-title":"Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors","volume":"38","author":"Aarons GA","year":"2011","unstructured":"Aarons GA, Hurlburt M, Horwitz SM. Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors. Adm Policy Ment Health. 2011;38(1):4\u201323. 10.1007\/s10488-010-0327-7.","journal-title":"Adm Policy Ment Health"},{"issue":"1","key":"ref21","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s13012-018-0842-6","article-title":"Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework","volume":"14","author":"Moullin JC","year":"2019","unstructured":"Moullin JC, Dickson KS, Stadnick NA, Rabin B, Aarons GA. Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Implement Sci. 2019;14(1):1. 10.1186\/s13012-018-0842-6.","journal-title":"Implement Sci"},{"issue":"6","key":"ref22","doi-asserted-by":"crossref","first-page":"527","DOI":"10.1037\/a0016830","article-title":"Toward a theory of motivational interviewing","volume":"64","author":"Miller WR","year":"2009","unstructured":"Miller WR, Rose GS. Toward a theory of motivational interviewing. Am Psychol. 2009;64(6):527.","journal-title":"Am Psychol"},{"key":"ref23","volume-title":"Motivational Interviewing: Helping People Change","author":"Miller WR","year":"2012","unstructured":"Miller WR, Rollnick S. Motivational Interviewing: Helping People Change. Guilford press; 2012."},{"key":"ref24","volume-title":"Motivational Interviewing in Groups","author":"Wagner CC","year":"2012","unstructured":"Wagner CC, Ingersoll KS. Motivational Interviewing in Groups. Guilford Press; 2012."},{"issue":"5","key":"ref25","doi-asserted-by":"crossref","first-page":"843","DOI":"10.1037\/0022-006X.71.5.843","article-title":"The efficacy of motivational interviewing: a meta-analysis of controlled clinical trials","volume":"71","author":"Burke BL","year":"2003","unstructured":"Burke BL, Arkowitz H, Menchola M. The efficacy of motivational interviewing: a meta-analysis of controlled clinical trials. J Consult Clin Psychol. 2003;71(5):843.","journal-title":"J Consult Clin Psychol"},{"issue":"3","key":"ref26","doi-asserted-by":"publisher","first-page":"328","DOI":"10.1093\/alcalc\/agl016","article-title":"EXCESSIVE DRINKING: A META-ANALYTIC REVIEW","volume":"41","author":"Vasilaki EI","year":"2006","unstructured":"Vasilaki EI, Hosier SG, Cox WM, THE EFFICACY OF MOTIVATIONAL INTERVIEWING AS A BRIEF INTERVENTION FOR. EXCESSIVE DRINKING: A META-ANALYTIC REVIEW. Alcohol Alcohol. 2006;41(3):328\u201335. 10.1093\/alcalc\/agl016.","journal-title":"Alcohol Alcohol"},{"issue":"2","key":"ref27","doi-asserted-by":"publisher","first-page":"137","DOI":"10.1177\/1049731509347850","article-title":"A Meta-Analysis of Motivational Interviewing: Twenty-Five Years of Empirical Studies","volume":"20","author":"Lundahl BW","year":"2010","unstructured":"Lundahl BW, Kunz C, Brownell C, Tollefson D, Burke BL. A Meta-Analysis of Motivational Interviewing: Twenty-Five Years of Empirical Studies. Res Social Work Pract. 2010;20(2):137\u201360. 10.1177\/1049731509347850.","journal-title":"Res Social Work Pract"},{"key":"ref28","volume-title":"Rewarding Provider Performance: Aligning Incentives in Medicare","author":"English WJ","year":"2008","unstructured":"English WJ. Rewarding Provider Performance: Aligning Incentives in Medicare. American College of Physicians; 2008."},{"issue":"1","key":"ref29","doi-asserted-by":"publisher","first-page":"449","DOI":"10.1146\/annurev-publhealth-032315-021457","article-title":"Impact of Provider Incentives on Quality and Value of Health Care","volume":"38","author":"Doran T","year":"2017","unstructured":"Doran T, Maurer KA, Ryan AM. Impact of Provider Incentives on Quality and Value of Health Care. Annu Rev Public Health. 2017;38(1):449\u201365. 10.1146\/annurev-publhealth-032315-021457.","journal-title":"Annu Rev Public Health"},{"key":"ref30","doi-asserted-by":"publisher","first-page":"107274","DOI":"10.1016\/j.ypmed.2022.107274","article-title":"The role of pay-for-performance in reducing healthcare disparities: A narrative literature review","volume":"164","author":"Conway A","year":"2022","unstructured":"Conway A, Satin D. The role of pay-for-performance in reducing healthcare disparities: A narrative literature review. Prev Med. 2022;164:107274. 10.1016\/j.ypmed.2022.107274.","journal-title":"Prev Med"},{"issue":"1","key":"ref31","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1186\/s13012-015-0209-1","article-title":"A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project","volume":"10","author":"Powell BJ","year":"2015","unstructured":"Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10(1):21. 10.1186\/s13012-015-0209-1.","journal-title":"Implement Sci"},{"issue":"5","key":"ref32","doi-asserted-by":"crossref","first-page":"777","DOI":"10.1037\/0021-9010.83.5.777","article-title":"Are financial incentives related to performance? A meta-analytic review of empirical research","volume":"83","author":"Jenkins GD","year":"1998","unstructured":"Jenkins GD Jr, Mitra A, Gupta N, Shaw JD. Are financial incentives related to performance? A meta-analytic review of empirical research. J Appl Psychol. 1998;83(5):777.","journal-title":"J Appl Psychol"},{"issue":"3","key":"ref33","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1111\/j.1937-8327.2003.tb00287.x","article-title":"The effects of incentives on workplace performance: A meta-analytic review of research studies 1","volume":"16","author":"Condly SJ","year":"2003","unstructured":"Condly SJ, Clark RE, Stolovitch HD. The effects of incentives on workplace performance: A meta-analytic review of research studies 1. Perform Improv Q. 2003;16(3):46\u201363.","journal-title":"Perform Improv Q"},{"key":"ref34","doi-asserted-by":"publisher","first-page":"1039375","DOI":"10.3389\/fpsyg.2022.1039375","article-title":"A cognitive evaluation and equity-based perspective of pay for performance on job performance: A meta-analysis and path model","volume":"13","author":"Chen Y","year":"2023","unstructured":"Chen Y, Zhang Z, Zhou J, Liu C, Zhang X, Yu T. A cognitive evaluation and equity-based perspective of pay for performance on job performance: A meta-analysis and path model. Front Psychol. 2023;13:1039375. 10.3389\/fpsyg.2022.1039375.","journal-title":"Front Psychol"},{"key":"ref35","author":"Kim JH","unstructured":"Kim JH, Gerhart B, Fang M. Do Financial Incentives Help or Harm Performance in Interesting Tasks?."},{"issue":"2\u20133","key":"ref36","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1016\/j.healthpol.2013.01.008","article-title":"Effects of pay for performance in health care: A systematic review of systematic reviews","volume":"110","author":"Eijkenaar F","year":"2013","unstructured":"Eijkenaar F, Emmert M, Scheppach M, Sch\u00f6ffski O. Effects of pay for performance in health care: A systematic review of systematic reviews. Health Policy. 2013;110(2\u20133):115\u201330. 10.1016\/j.healthpol.2013.01.008.","journal-title":"Health Policy"},{"issue":"S1","key":"ref37","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1007\/s11606-015-3567-0","article-title":"Implementation Processes and Pay for Performance in Healthcare: A Systematic Review","volume":"31","author":"Kondo KK","year":"2016","unstructured":"Kondo KK, Damberg CL, Mendelson A, et al. Implementation Processes and Pay for Performance in Healthcare: A Systematic Review. J GEN INTERN MED. 2016;31(S1):61\u20139. 10.1007\/s11606-015-3567-0.","journal-title":"J GEN INTERN MED"},{"issue":"7","key":"ref38","doi-asserted-by":"publisher","first-page":"592","DOI":"10.1016\/j.healthpol.2022.03.007","article-title":"Effectiveness of pay for performance to non-physician health care providers: A systematic review","volume":"126","author":"Asadi-Aliabadi M","year":"2022","unstructured":"Asadi-Aliabadi M, Karimi SM, Tehrani-Banihashemi A, et al. Effectiveness of pay for performance to non-physician health care providers: A systematic review. Health Policy. 2022;126(7):592\u2013602. 10.1016\/j.healthpol.2022.03.007.","journal-title":"Health Policy"},{"key":"ref39","doi-asserted-by":"publisher","DOI":"10.1186\/1748-5908-5-5","author":"Garner BR","year":"2010","unstructured":"Garner BR, Godley SH, Dennis ML, Godley MD, Shepard DS. The Reinforcing Therapist Performance (RTP) experiment: Study protocol for a cluster randomized trial. Implement Sci. 2010;5. doi:Artn 5 10.1186\/1748-5908-5-5."},{"issue":"10","key":"ref40","doi-asserted-by":"publisher","first-page":"938","DOI":"10.1001\/archpediatrics.2012.802","article-title":"Using Pay for Performance to Improve Treatment Implementation for Adolescent Substance Use Disorders Results From a Cluster Randomized Trial","volume":"166","author":"Garner BR","year":"2012","unstructured":"Garner BR, Godley SH, Dennis ML, Hunter BD, Bair CML, Godley MD. Using Pay for Performance to Improve Treatment Implementation for Adolescent Substance Use Disorders Results From a Cluster Randomized Trial. Arch Pediat Adol Med. 2012;166(10):938\u201344. 10.1001\/archpediatrics.2012.802.","journal-title":"Arch Pediat Adol Med"},{"key":"ref41","doi-asserted-by":"publisher","first-page":"ARTN","DOI":"10.1186\/s13012-018-0774-1","article-title":"Pay-for-performance as a cost-effective implementation strategy: results from a cluster randomized trial","volume":"13","author":"Garner BR","year":"2018","unstructured":"Garner BR, Lwin AK, Strickler GK, Hunter BD, Shepard DS. Pay-for-performance as a cost-effective implementation strategy: results from a cluster randomized trial. Implement Sci. 2018;13:ARTN. 92 10.1186\/s13012-018-0774-1.","journal-title":"Implement Sci"},{"issue":"1","key":"ref42","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1186\/s13012-017-0626-4","article-title":"Quality improvement, implementation, and dissemination strategies to improve mental health care for children and adolescents: a systematic review","volume":"12","author":"Forman-Hoffman VL","year":"2017","unstructured":"Forman-Hoffman VL, Middleton JC, McKeeman JL, et al. Quality improvement, implementation, and dissemination strategies to improve mental health care for children and adolescents: a systematic review. Implement Sci. 2017;12(1):93. 10.1186\/s13012-017-0626-4.","journal-title":"Implement Sci"},{"issue":"9","key":"ref43","doi-asserted-by":"crossref","first-page":"936","DOI":"10.1016\/j.jclinepi.2010.12.003","article-title":"Systematic review of stepped wedge cluster randomized trials shows that design is particularly used to evaluate interventions during routine implementation","volume":"64","author":"Mdege ND","year":"2011","unstructured":"Mdege ND, Man MS, Taylor CA, Torgerson DJ. Systematic review of stepped wedge cluster randomized trials shows that design is particularly used to evaluate interventions during routine implementation. J Clin Epidemiol. 2011;64(9):936\u201348.","journal-title":"J Clin Epidemiol"},{"issue":"12","key":"ref44","doi-asserted-by":"crossref","first-page":"1249","DOI":"10.1016\/j.jclinepi.2012.06.004","article-title":"Use of the stepped wedge design cannot be recommended: a critical appraisal and comparison with the classic cluster randomized controlled trial design","volume":"65","author":"Kotz D","year":"2012","unstructured":"Kotz D, Spigt M, Arts IC, Crutzen R, Viechtbauer W. Use of the stepped wedge design cannot be recommended: a critical appraisal and comparison with the classic cluster randomized controlled trial design. J Clin Epidemiol. 2012;65(12):1249\u201352.","journal-title":"J Clin Epidemiol"},{"issue":"12","key":"ref45","doi-asserted-by":"crossref","first-page":"1255","DOI":"10.1016\/j.jclinepi.2012.06.016","article-title":"Researchers should convince policy makers to perform a classic cluster randomized controlled trial instead of a stepped wedge design when an intervention is rolled out","volume":"65","author":"Kotz D","year":"2012","unstructured":"Kotz D, Spigt M, Arts IC, Crutzen R, Viechtbauer W. Researchers should convince policy makers to perform a classic cluster randomized controlled trial instead of a stepped wedge design when an intervention is rolled out. J Clin Epidemiol. 2012;65(12):1255\u20136.","journal-title":"J Clin Epidemiol"},{"issue":"9","key":"ref46","doi-asserted-by":"crossref","first-page":"1059","DOI":"10.1016\/j.jclinepi.2013.05.004","article-title":"The stepped wedge design does not inherently have more power than a cluster randomized controlled trial","volume":"66","author":"Kotz D","year":"2013","unstructured":"Kotz D, Spigt M, Arts IC, Crutzen R, Viechtbauer W. The stepped wedge design does not inherently have more power than a cluster randomized controlled trial. J Clin Epidemiol. 2013;66(9):1059\u201360.","journal-title":"J Clin Epidemiol"},{"issue":"1","key":"ref47","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1007\/s10488-008-0197-4","article-title":"Implementation Research in Mental Health Services: an Emerging Science with Conceptual, Methodological, and Training challenges","volume":"36","author":"Proctor EK","year":"2009","unstructured":"Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation Research in Mental Health Services: an Emerging Science with Conceptual, Methodological, and Training challenges. Adm Policy Ment Health. 2009;36(1):24\u201334. 10.1007\/s10488-008-0197-4.","journal-title":"Adm Policy Ment Health"},{"issue":"3","key":"ref48","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1097\/MLR.0b013e3182408812","article-title":"Effectiveness-implementation Hybrid Designs: Combining Elements of Clinical Effectiveness and Implementation Research to Enhance Public Health Impact","volume":"50","author":"Curran GM","year":"2012","unstructured":"Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation Hybrid Designs: Combining Elements of Clinical Effectiveness and Implementation Research to Enhance Public Health Impact. Med Care. 2012;50(3):217\u201326. 10.1097\/MLR.0b013e3182408812.","journal-title":"Med Care"},{"issue":"1","key":"ref49","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1186\/s13012-015-0240-2","article-title":"Implementation science: a reappraisal of our journal mission and scope","volume":"10","author":"Foy R","year":"2015","unstructured":"Foy R, Sales A, Wensing M, et al. Implementation science: a reappraisal of our journal mission and scope. Implement Sci. 2015;10(1):51. 10.1186\/s13012-015-0240-2.","journal-title":"Implement Sci"},{"issue":"1","key":"ref50","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1186\/s13012-020-01041-8","article-title":"The Implementation Research Logic Model: a method for planning, executing, reporting, and synthesizing implementation projects","volume":"15","author":"Smith JD","year":"2020","unstructured":"Smith JD, Li DH, Rafferty MR. The Implementation Research Logic Model: a method for planning, executing, reporting, and synthesizing implementation projects. Implement Sci. 2020;15(1):84. 10.1186\/s13012-020-01041-8.","journal-title":"Implement Sci"},{"key":"ref51","doi-asserted-by":"publisher","first-page":"1053496","DOI":"10.3389\/frhs.2022.1053496","article-title":"Reflections on 10 years of effectiveness-implementation hybrid studies","volume":"2","author":"Curran GM","year":"2022","unstructured":"Curran GM, Landes SJ, McBain SA, et al. Reflections on 10 years of effectiveness-implementation hybrid studies. FrontHealth Serv. 2022;2:1053496. 10.3389\/frhs.2022.1053496.","journal-title":"FrontHealth Serv"},{"key":"ref52","doi-asserted-by":"publisher","first-page":"263348952210892","DOI":"10.1177\/26334895221089266","article-title":"The implementation & sustainment facilitation (ISF) strategy: Cost and cost-effectiveness results from a 39-site cluster randomized trial integrating substance use services in community-based HIV service organizations","volume":"3","author":"Hinde JM","year":"2022","unstructured":"Hinde JM, Garner BR, Watson CJ, Ramanan R, Ball EL, Tueller SJ. The implementation & sustainment facilitation (ISF) strategy: Cost and cost-effectiveness results from a 39-site cluster randomized trial integrating substance use services in community-based HIV service organizations. Implement Res Pract. 2022;3:263348952210892. 10.1177\/26334895221089266.","journal-title":"Implement Res Pract"},{"key":"ref53","author":"Association D","year":"2013","unstructured":"American Psychiatric Association, Association D. AP. Diagnostic and Statistical Manual of Mental Disorders: DSM-5. Vol\u00a05. American psychiatric association Washington, DC; 2013."},{"issue":"2","key":"ref54","doi-asserted-by":"crossref","first-page":"318","DOI":"10.1037\/pst0000221","article-title":"Design feasibility of an automated, machine-learning based feedback system for motivational interviewing","volume":"56","author":"Imel ZE","year":"2019","unstructured":"Imel ZE, Pace BT, Soma CS, et al. Design feasibility of an automated, machine-learning based feedback system for motivational interviewing. Psychotherapy. 2019;56(2):318.","journal-title":"Psychotherapy"},{"issue":"3","key":"ref55","doi-asserted-by":"publisher","first-page":"324","DOI":"10.1007\/s11414-013-9381-8","article-title":"Parallel Processes: Using Motivational Interviewing as an Implementation Coaching Strategy","volume":"41","author":"Hettema JE","year":"2014","unstructured":"Hettema JE, Ernst D, Williams JR, Miller KJ. Parallel Processes: Using Motivational Interviewing as an Implementation Coaching Strategy. J Behav Health Serv Res. 2014;41(3):324\u201336. 10.1007\/s11414-013-9381-8.","journal-title":"J Behav Health Serv Res"},{"issue":"1","key":"ref56","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/1748-5908-5-75","article-title":"Employing external facilitation to implement cognitive behavioral therapy in VA clinics: a pilot study","volume":"5","author":"Kauth MR","year":"2010","unstructured":"Kauth MR, Sullivan G, Blevins D, et al. Employing external facilitation to implement cognitive behavioral therapy in VA clinics: a pilot study. Implement Sci. 2010;5(1):1\u201311.","journal-title":"Implement Sci"},{"issue":"3","key":"ref57","doi-asserted-by":"crossref","first-page":"322","DOI":"10.1016\/j.ajog.2017.12.005","article-title":"A randomized controlled trial of screening and brief interventions for substance misuse in reproductive health","volume":"218","author":"Martino S","year":"2018","unstructured":"Martino S, Ondersma SJ, Forray A, et al. A randomized controlled trial of screening and brief interventions for substance misuse in reproductive health. Am J Obstet Gynecol. 2018;218(3):322. e1-322. e12.","journal-title":"Am J Obstet Gynecol"},{"key":"ref58","doi-asserted-by":"publisher","first-page":"64","DOI":"10.3389\/fpubh.2019.00064","article-title":"RE-AIM Planning and Evaluation Framework: Adapting to New Science and Practice With a 20-Year Review","volume":"7","author":"Glasgow RE","year":"2019","unstructured":"Glasgow RE, Harden SM, Gaglio B, et al. RE-AIM Planning and Evaluation Framework: Adapting to New Science and Practice With a 20-Year Review. Front Public Health. 2019;7:64. 10.3389\/fpubh.2019.00064.","journal-title":"Front Public Health"},{"issue":"2","key":"ref59","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1007\/s10488-010-0319-7","article-title":"Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda","volume":"38","author":"Proctor E","year":"2011","unstructured":"Proctor E, Silmere H, Raghavan R, et al. Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda. Adm Policy Ment Health. 2011;38(2):65\u201376. 10.1007\/s10488-010-0319-7.","journal-title":"Adm Policy Ment Health"},{"issue":"1","key":"ref60","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/S0272-7358(97)00043-3","article-title":"Program integrity in primary and early secondary prevention: are implementation effects out of control?","volume":"18","author":"Dane AV","year":"1998","unstructured":"Dane AV, Schneider BH. Program integrity in primary and early secondary prevention: are implementation effects out of control? Clin Psychol Rev. 1998;18(1):23\u201345.","journal-title":"Clin Psychol Rev"},{"issue":"10","key":"ref61","doi-asserted-by":"crossref","first-page":"1092","DOI":"10.1001\/archinte.166.10.1092","article-title":"A brief measure for assessing generalized anxiety disorder: the GAD-7","volume":"166","author":"Spitzer RL","year":"2006","unstructured":"Spitzer RL, Kroenke K, Williams JB, L\u00f6we B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092\u20137.","journal-title":"Arch Intern Med"},{"key":"ref62","unstructured":"SAS version 9.4. (2023). SAS Institute Inc. Cary, NC, USA."},{"issue":"427","key":"ref63","doi-asserted-by":"crossref","first-page":"846","DOI":"10.1080\/01621459.1994.10476818","article-title":"Estimation of regression coefficients when some regressors are not always observed","volume":"89","author":"Robins JM","year":"1994","unstructured":"Robins JM, Rotnitzky A, Zhao LP. Estimation of regression coefficients when some regressors are not always observed. J Am Stat Assoc. 1994;89(427):846\u201366.","journal-title":"J Am Stat Assoc"},{"issue":"4","key":"ref64","doi-asserted-by":"crossref","first-page":"962","DOI":"10.1111\/j.1541-0420.2005.00377.x","article-title":"Doubly robust estimation in missing data and causal inference models","volume":"61","author":"Bang H","year":"2005","unstructured":"Bang H, Robins JM. Doubly robust estimation in missing data and causal inference models. Biometrics. 2005;61(4):962\u201373.","journal-title":"Biometrics"},{"issue":"7","key":"ref65","doi-asserted-by":"publisher","first-page":"761","DOI":"10.1093\/aje\/kwq439","article-title":"Doubly Robust Estimation of Causal Effects","volume":"173","author":"Funk MJ","year":"2011","unstructured":"Funk MJ, Westreich D, Wiesen C, St\u00fcrmer T, Brookhart MA, Davidian M. Doubly Robust Estimation of Causal Effects. Am J Epidemiol. 2011;173(7):761\u20137. 10.1093\/aje\/kwq439.","journal-title":"Am J Epidemiol"},{"issue":"511","key":"ref66","doi-asserted-by":"crossref","first-page":"1024","DOI":"10.1080\/01621459.2014.958155","article-title":"Bias-reduced doubly robust estimation","volume":"110","author":"Vermeulen K","year":"2015","unstructured":"Vermeulen K, Vansteelandt S. Bias-reduced doubly robust estimation. J Am Stat Assoc. 2015;110(511):1024\u201336.","journal-title":"J Am Stat Assoc"},{"issue":"1","key":"ref67","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1186\/s13722-019-0150-8","article-title":"HIV care provider perceptions and approaches to managing unhealthy alcohol use in primary HIV care settings: a qualitative study","volume":"14","author":"Chichetto NE","year":"2019","unstructured":"Chichetto NE, Mannes ZL, Allen MK, Cook RL, Ennis N. HIV care provider perceptions and approaches to managing unhealthy alcohol use in primary HIV care settings: a qualitative study. Addict Sci Clin Pract. 2019;14(1):21. 10.1186\/s13722-019-0150-8.","journal-title":"Addict Sci Clin Pract"},{"key":"ref68","first-page":"1","article-title":"Serving the co-morbid mental health and substance use needs of people with HIV","volume":"2021","author":"Conway FN","unstructured":"Conway FN, Rountree MA, Jones KV. Serving the co-morbid mental health and substance use needs of people with HIV. Community Mental Health Journal Published online 2021:1\u201312.","journal-title":"Community Mental Health Journal Published online"},{"issue":"8","key":"ref69","doi-asserted-by":"publisher","first-page":"ofac365","DOI":"10.1093\/ofid\/ofac365","article-title":"Interventions for Integrating Behavioral Health Services Into HIV Clinical Care: A Narrative Review","volume":"9","author":"Goldhammer H","year":"2022","unstructured":"Goldhammer H, Marc LG, Chavis NS, et al. Interventions for Integrating Behavioral Health Services Into HIV Clinical Care: A Narrative Review. Open Forum Infectious Diseases. 2022;9(8):ofac365. 10.1093\/ofid\/ofac365.","journal-title":"Open Forum Infectious Diseases"},{"issue":"1","key":"ref70","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1186\/s13011-017-0104-7","article-title":"Strategies to facilitate integrated care for people with alcohol and other drug problems: a systematic review","volume":"12","author":"Savic M","year":"2017","unstructured":"Savic M, Best D, Manning V, Lubman DI. Strategies to facilitate integrated care for people with alcohol and other drug problems: a systematic review. Subst Abuse Treat Prev Policy. 2017;12(1):19. 10.1186\/s13011-017-0104-7.","journal-title":"Subst Abuse Treat Prev Policy"},{"key":"ref71","article-title":"Cognitive evaluation theory and some comments on the Calder and Staw critique","author":"Deci EL","unstructured":"Deci EL, Cascio WF, Krusell J. Cognitive evaluation theory and some comments on the Calder and Staw critique. Published online 1975."},{"issue":"6","key":"ref72","doi-asserted-by":"crossref","first-page":"627","DOI":"10.1037\/0033-2909.125.6.627","article-title":"A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation","volume":"125","author":"Deci EL","year":"1999","unstructured":"Deci EL, Koestner R, Ryan RM. A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychol Bull. 1999;125(6):627.","journal-title":"Psychol Bull"},{"issue":"2","key":"ref73","doi-asserted-by":"crossref","first-page":"147","DOI":"10.2224\/sbp.1988.16.2.147","article-title":"Cognitive evaluation theory: A meta-analytic review of the literature","volume":"16","author":"Rummel A","year":"1988","unstructured":"Rummel A, Feinberg R. Cognitive evaluation theory: A meta-analytic review of the literature. Social Behav Personality: Int J. 1988;16(2):147\u201364.","journal-title":"Social Behav Personality: Int J"},{"issue":"2","key":"ref74","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1016\/0092-6566(85)90023-6","article-title":"The general causality orientations scale: Self-determination in personality","volume":"19","author":"Deci EL","year":"1985","unstructured":"Deci EL, Ryan RM. The general causality orientations scale: Self-determination in personality. J Res Pers. 1985;19(2):109\u201334.","journal-title":"J Res Pers"},{"key":"ref75","author":"Deci EL","unstructured":"Deci EL, Ryan RM. Self-determination theory. Published online 2012."},{"key":"ref76","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1146\/annurev-orgpsych-032516-113108","article-title":"Self-determination theory in work organizations: The state of a science","volume":"4","author":"Deci EL","year":"2017","unstructured":"Deci EL, Olafsen AH, Ryan RM. Self-determination theory in work organizations: The state of a science. Annual Rev organizational Psychol organizational Behav. 2017;4:19\u201343.","journal-title":"Annual Rev organizational Psychol organizational Behav"},{"key":"ref77","author":"Ryan RM","unstructured":"Ryan RM, Deci EL. Self-determination theory. Basic psychological needs in motivation, development, and wellness. Published online 2017."},{"issue":"2","key":"ref78","doi-asserted-by":"crossref","first-page":"565","DOI":"10.2466\/21.02.PR0.115c23z1","article-title":"Effect of client-therapist gender match on the therapeutic relationship: An exploratory analysis","volume":"115","author":"Bhati KS","year":"2014","unstructured":"Bhati KS. Effect of client-therapist gender match on the therapeutic relationship: An exploratory analysis. Psychol Rep. 2014;115(2):565\u201383.","journal-title":"Psychol Rep"}],"container-title":[],"original-title":[],"link":[{"URL":"https:\/\/www.researchsquare.com\/article\/rs-3685134\/v1","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.researchsquare.com\/article\/rs-3685134\/v1.html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,20]],"date-time":"2023-12-20T17:46:01Z","timestamp":1703094361000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.researchsquare.com\/article\/rs-3685134\/v1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,20]]},"references-count":78,"URL":"https:\/\/doi.org\/10.21203\/rs.3.rs-3685134\/v1","relation":{},"subject":[],"published":{"date-parts":[[2023,12,20]]},"subtype":"preprint"}}