{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,1]],"date-time":"2026-08-01T00:45:06Z","timestamp":1785545106408,"version":"3.56.0"},"reference-count":43,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J Med Internet Res"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Digital health interventions (DHIs) can broaden the reach of HIV prevention interventions and overcome barriers for young cisgender men who have sex with men (YMSM). Community-based organizations (CBOs) have delivered HIV prevention interventions for decades, but few studies have examined how CBOs implement DHIs, including recruitment and retention. Keep It Up! (KIU!) is a Centers for Disease Control and Prevention\u2013designated best-evidence DHI that can promote risk reduction behaviors and reduce sexually transmitted infection incidence.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n We sought to descriptively assess CBOs\u2019 approaches to recruitment and retention, elucidate lessons learned, and provide examples of recruitment and retention for future implementers.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n Twenty-two CBOs in counties with high HIV rates and large proportions of YMSM were selected through a request for proposal process to implement KIU!. Data were extracted from request for proposal applications and notes from monthly calls with CBO staff. Twenty-five staff members across CBOs were interviewed in the middle of implementation. A descriptive thematic analysis of the lessons learned in recruiting and retaining participants was performed. The research team developed an application dashboard for CBO staff to register participants, track participant progress through the intervention, record and track participant contact, and export usage data. CBO logins to the KIU! dashboard were tracked across the implementation. To descriptively compare approaches to recruitment and retention, the study team divided CBOs according to annual HIV testing volume in the 3 years prior to implementation and years of HIV service provision to YMSM.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n The most frequent modes of recruitment were outreach and community partnerships (21\/22, 95%), and the least frequent modes were via hook-up apps (6\/22, 27%) and participant referrals (5\/22, 23%). CBOs with a low HIV testing volume used online recruitment slightly more frequently, while medium-volume CBOs most frequently used hook-up apps for recruitment. Low-volume CBOs more frequently used phone calls and emails to remind participants to complete intervention modules, while high- and medium-volume CBOs more frequently used text messages. CBOs with more years of HIV service provision to YMSM more frequently had a set reminder schedule for contact with participants. CBO staff identified a need to change how KIU! is pitched to clients by using personalized, recipient-centered language rather than technical jargon. CBOs changed intake forms to ensure that staff remembered to offer KIU! to participants. CBOs had a difficult time retaining participants despite holding in-person events for enrolled participants (eg, raffles and trivia nights) and altering the frequency with which they reminded participants to complete modules.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Although CBOs had experience in implementing other evidence-based interventions with YMSM, there was no consensus on successful recruitment and retention strategies for this population. We have presented approaches that future CBOs may use in their own implementation of KIU!.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/63199","type":"journal-article","created":{"date-parts":[[2025,11,3]],"date-time":"2025-11-03T22:11:13Z","timestamp":1762207873000},"page":"e63199-e63199","source":"Crossref","is-referenced-by-count":0,"title":["Community-Based Organizations\u2019 Approaches to Recruitment and Retention for a Digital HIV Prevention Intervention for Young Men Who Have Sex With Men: A Mixed Methods Study"],"prefix":"10.2196","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8030-2775","authenticated-orcid":false,"given":"Alithia","family":"Zamantakis","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2792-7140","authenticated-orcid":false,"given":"Elizabeth","family":"Danielson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-7689-816X","authenticated-orcid":false,"given":"Emma","family":"Rudd","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1996-6863","authenticated-orcid":false,"given":"J Pablo","family":"Zapata","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6715-1530","authenticated-orcid":false,"given":"Nanette","family":"Benbow","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2131-643X","authenticated-orcid":false,"given":"Rana","family":"Saber","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0073-1466","authenticated-orcid":false,"given":"Ashley A","family":"Knapp","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9222-5116","authenticated-orcid":false,"given":"Brian","family":"Mustanski","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2025,11,3]]},"reference":[{"issue":"8","key":"R1","doi-asserted-by":"publisher","first-page":"945","DOI":"10.1080\/09540121.2012.668167","article-title":"eHealth interventions for HIV prevention","volume":"24","author":"Noar","journal-title":"AIDS Care"},{"issue":"6","key":"R2","doi-asserted-by":"publisher","first-page":"403","DOI":"10.1007\/s11904-018-0415-y","article-title":"Implementation of eHealth interventions across the HIV care cascade: a review of recent research","volume":"15","author":"Kemp","journal-title":"Curr HIV\/AIDS Rep"},{"issue":"4","key":"R3","doi-asserted-by":"publisher","first-page":"335","DOI":"10.1007\/s11904-019-00455-4","article-title":"Design considerations for implementing eHealth behavioral interventions for HIV prevention in evolving sociotechnical landscapes","volume":"16","author":"Li","journal-title":"Curr HIV\/AIDS Rep"},{"issue":"6","key":"R4","doi-asserted-by":"publisher","first-page":"478","DOI":"10.1177\/0963721413495872","article-title":"Realizing the potential of behavioral intervention technologies","volume":"22","author":"Schueller","journal-title":"Curr Dir Psychol Sci"},{"issue":"2","key":"R5","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.1001382","article-title":"Scaling up mHealth: where is the evidence?","volume":"10","author":"Tomlinson","journal-title":"PLoS Med"},{"issue":"3 Suppl A","key":"R6","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1521\/aeap.14.4.27.23886","article-title":"Factors affecting the delivery of HIV\/AIDS prevention programs by community-based organizations","volume":"14","author":"Chillag","journal-title":"AIDS Educ Prev"},{"issue":"2","key":"R7","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1521\/aeap.2011.23.2.105","article-title":"Providers\u2019 perceptions of and receptivity toward evidence-based HIV prevention interventions","volume":"23","author":"Owczarzak","journal-title":"AIDS Educ Prev"},{"issue":"1","key":"R8","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1093\/epirev\/mxac008","article-title":"Addressing health inequities in digital clinical trials: a review of challenges and solutions from the field of HIV research","volume":"44","author":"Wirtz","journal-title":"Epidemiol Rev"},{"issue":"4","key":"R9","doi-asserted-by":"publisher","first-page":"451","DOI":"10.1007\/s11904-015-0291-7","article-title":"Recent evidence for emerging digital technologies to support global HIV engagement in care","volume":"12","author":"Jongbloed","journal-title":"Curr HIV\/AIDS Rep"},{"issue":"1","key":"R10","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-016-0359-3","article-title":"Understanding factors affecting patient and public engagement and recruitment to digital health interventions: a systematic review of qualitative studies","volume":"16","author":"O\u2019Connor","journal-title":"BMC Med Inform Decis Mak"},{"issue":"8","key":"R11","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0220060","article-title":"A need for implementation science to optimise the use of evidence-based interventions in HIV care: a systematic literature review","volume":"14","author":"Cox","journal-title":"PLoS ONE"},{"issue":"3","key":"R12","doi-asserted-by":"publisher","first-page":"152","DOI":"10.1007\/s11904-024-00693-1","article-title":"\u201cYour package could not be delivered\u201d: the state of digital HIV intervention implementation in the US","volume":"21","author":"Li","journal-title":"Curr HIV\/AIDS Rep"},{"issue":"5","key":"R13","doi-asserted-by":"publisher","first-page":"779","DOI":"10.1111\/chso.12533","article-title":"Incentives matter: an experiment in participant engagement from a family\u2010centred adolescent HIV prevention programme in South Africa","volume":"36","author":"Spyrelis","journal-title":"Children & Society"},{"issue":"2","key":"R14","doi-asserted-by":"publisher","DOI":"10.3390\/ijerph17020399","article-title":"A community-based participatory approach to the development and implementation of an HIV health behavior intervention: lessons learned in navigating research and practice systems from Project HAPPY","volume":"17","author":"Holliday","journal-title":"Int J Environ Res Public Health"},{"issue":"5","key":"R15","doi-asserted-by":"publisher","first-page":"471","DOI":"10.1097\/PHH.0000000000001020","article-title":"Integrating financial incentives for viral load suppression into HIV care coordination programs: considerations for development and implementation","volume":"26","author":"Gambone","journal-title":"J Public Health Manag Pract"},{"issue":"1","key":"R16","doi-asserted-by":"publisher","DOI":"10.1186\/s13012-020-01075-y","article-title":"Designing provider-focused implementation trials with purpose and intent: introducing the PRECIS-2-PS tool","volume":"16","author":"Norton","journal-title":"Implement Sci"},{"key":"R17","doi-asserted-by":"publisher","DOI":"10.1186\/1745-6215-10-37","article-title":"Making trials matter: pragmatic and explanatory trials and the problem of applicability","volume":"10","author":"Treweek","journal-title":"Trials"},{"issue":"1","key":"R18","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1111\/j.1752-8062.2011.00383.x","article-title":"Developing robust, sustainable, implementation systems using rigorous, rapid and relevant science","volume":"5","author":"Glasgow","journal-title":"Clin Transl Sci"},{"issue":"3","key":"R19","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1177\/1090198113486805","article-title":"What does it mean to be pragmatic? Pragmatic methods, measures, and models to facilitate research translation","volume":"40","author":"Glasgow","journal-title":"Health Educ Behav"},{"key":"R20","doi-asserted-by":"publisher","DOI":"10.1186\/s13012-014-0096-x","article-title":"How pragmatic is it? Lessons learned using PRECIS and RE-AIM for determining pragmatic characteristics of research","volume":"9","author":"Gaglio","journal-title":"Implement Sci"},{"key":"R21","doi-asserted-by":"publisher","DOI":"10.1016\/j.psychres.2019.112513","article-title":"An introduction to effectiveness-implementation hybrid designs","volume":"280","author":"Landes","journal-title":"Psychiatry Res"},{"key":"R22","doi-asserted-by":"publisher","DOI":"10.1016\/j.cct.2023.107134","article-title":"Keep It Up! 3.0: study protocol for a type III hybrid implementation-effectiveness cluster-randomized trial","volume":"127","author":"Mustanski","journal-title":"Contemp Clin Trials"},{"issue":"9","key":"R23","doi-asserted-by":"publisher","first-page":"2999","DOI":"10.1007\/s10461-013-0507-z","article-title":"Feasibility, acceptability, and preliminary efficacy of an online HIV prevention program for diverse young men who have sex with men: the keep it up! intervention","volume":"17","author":"Mustanski","journal-title":"AIDS Behav"},{"issue":"2","key":"R24","doi-asserted-by":"publisher","first-page":"289","DOI":"10.1007\/s10508-009-9596-1","article-title":"Internet use and sexual health of young men who have sex with men: a mixed-methods study","volume":"40","author":"Mustanski","journal-title":"Arch Sex Behav"},{"issue":"2","key":"R25","doi-asserted-by":"publisher","first-page":"151","DOI":"10.1016\/j.amepre.2018.04.026","article-title":"Biomedical and behavioral outcomes of Keep It Up!: an eHealth HIV prevention program RCT","volume":"55","author":"Mustanski","journal-title":"Am J Prev Med"},{"issue":"3","key":"R26","doi-asserted-by":"publisher","first-page":"231","DOI":"10.1521\/aeap.2016.28.3.231","article-title":"Implementation and evaluation of the Keep It Up! online HIV prevention intervention in a community-based setting","volume":"28","author":"Greene","journal-title":"AIDS Educ Prev"},{"issue":"3","key":"R27","doi-asserted-by":"publisher","first-page":"238","DOI":"10.1037\/\/0278-6133.13.3.238","article-title":"Empirical tests of an information-motivation-behavioral skills model of AIDS-preventive behavior with gay men and heterosexual university students","volume":"13","author":"Fisher","journal-title":"Health Psychol"},{"issue":"4","key":"R28","doi-asserted-by":"publisher","first-page":"1208","DOI":"10.1007\/s10461-016-1446-2","article-title":"The information\u2013motivation\u2013behavioral skills model revisited: a network-perspective structural equation model within a public sexually transmitted infection clinic sample of hazardous alcohol users","volume":"21","author":"John","journal-title":"AIDS Behav"},{"issue":"1","key":"R29","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1080\/15381501.2011.549064","article-title":"Are information, motivation, and behavioral skills linked with HIV-related sexual risk among young men who have sex with men?","volume":"10","author":"Fisher","journal-title":"J HIV AIDS Soc Serv"},{"issue":"8","key":"R30","doi-asserted-by":"publisher","first-page":"483","DOI":"10.1097\/OLQ.0000000000000636","article-title":"Prevalence and risk factors for rectal and urethral sexually transmitted infections from self-collected samples among young men who have sex with men participating in the Keep It Up! 2.0 randomized controlled trial","volume":"44","author":"Mustanski","journal-title":"Sex Transm Dis"},{"key":"R31","doi-asserted-by":"publisher","DOI":"10.1186\/1748-5908-4-50","article-title":"Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science","volume":"4","author":"Damschroder","journal-title":"Implement Sci"},{"key":"R32","unstructured":"Glaser BG Strauss AL . The Discovery of Grounded Theory: Strategies for Qualitative Research. Sociology Press; 1967. ISBN: 9780202302607"},{"key":"R33","doi-asserted-by":"publisher","DOI":"10.1007\/s10461-025-04838-0","article-title":"Comparing implementation and effectiveness outcomes for two implementation strategies of the Keep It Up! digital HIV prevention program: a type 3 hybrid effectiveness-implementation trial","author":"Mustanski","journal-title":"AIDS Behav"},{"issue":"5","key":"R34","doi-asserted-by":"publisher","first-page":"1479","DOI":"10.1007\/s10461-025-04619-9","article-title":"Substance use classes among YMSM in an HIV digital health intervention program: implications for acceptability, engagement, and health outcomes","volume":"29","author":"Zapata","journal-title":"AIDS Behav"},{"issue":"2","key":"R35","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1097\/OLQ.0b013e3181f0c051","article-title":"Effectiveness of the VOICES\/VOCES sexually transmitted disease\/human immunodeficiency virus prevention intervention when administered by health department staff: does it work in the \u201creal world\u201d?","volume":"38","author":"Neumann","journal-title":"Sex Transm Dis"},{"issue":"Supplement 2","key":"R36","doi-asserted-by":"publisher","first-page":"S68","DOI":"10.1097\/01.qai.0000140604.57478.67","article-title":"Prevention for substance-using HIV-positive young people","volume":"37","author":"Rotheram-Borus","journal-title":"J Acquir Immune Defic Syndr"},{"issue":"8","key":"R37","doi-asserted-by":"publisher","first-page":"1129","DOI":"10.2105\/ajph.86.8_pt_1.1129","article-title":"The Mpowerment Project: a community-level HIV prevention intervention for young gay men","volume":"86","author":"Kegeles","journal-title":"Am J Public Health"},{"issue":"2","key":"R38","doi-asserted-by":"publisher","first-page":"113","DOI":"10.1353\/cpr.2011.0016","article-title":"Community-centered family health history: a customized approach to increased health communication and awareness","volume":"5","author":"O\u2019Leary","journal-title":"Prog Community Health Partnersh"},{"issue":"4","key":"R39","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1016\/j.apnr.2014.02.007","article-title":"Strategies to enhance participant recruitment and retention in research involving a community-based population","volume":"27","author":"McCullagh","journal-title":"Appl Nurs Res"},{"issue":"1","key":"R40","doi-asserted-by":"publisher","DOI":"10.1186\/s12874-017-0310-z","article-title":"Participant retention practices in longitudinal clinical research studies with high retention rates","volume":"17","author":"Abshire","journal-title":"BMC Med Res Methodol"},{"issue":"1","key":"R41","doi-asserted-by":"publisher","DOI":"10.1186\/s12874-018-0586-7","article-title":"Retention strategies in longitudinal cohort studies: a systematic review and meta-analysis","volume":"18","author":"Teague","journal-title":"BMC Med Res Methodol"},{"issue":"1","key":"R42","doi-asserted-by":"publisher","DOI":"10.1186\/1471-2458-11-249","article-title":"A systematic review of the effect of retention methods in population-based cohort studies","volume":"11","author":"Booker","journal-title":"BMC Public Health"},{"key":"R43","unstructured":"Mustanski B . Results from the keep it up! 2.0 RCT: an ehealth intervention significantly reduces biological and behavioral markers of HIV risk in diverse young MSM. In: Centers for Disease Control and Prevention Division of HIV Prevention Translating Research into Practice (TRIP) Seminar Series. Centers for Disease Control and Prevention; 2017."}],"container-title":["Journal of Medical Internet Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.jmir.org\/2025\/1\/e63199","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T22:31:12Z","timestamp":1785537072000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.jmir.org\/2025\/1\/e63199"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,3]]},"references-count":43,"URL":"https:\/\/doi.org\/10.2196\/63199","relation":{},"ISSN":["1438-8871"],"issn-type":[{"value":"1438-8871","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,11,3]]},"article-number":"v27i8e63199"}}