{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,7]],"date-time":"2026-08-07T16:05:59Z","timestamp":1786118759330,"version":"build-2736575974"},"reference-count":57,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Form Res"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n While technology can widen access to mental health treatments, digital mental health interventions (DMHIs) frequently have low engagement and high dropout rates. A better understanding of user engagement with DMHIs can help researchers design technologies that users are more likely to benefit from. However, a major challenge is that the term \u201cengagement\u201d is very broad, not well-understood, and operationalized differently across projects. Few studies have explored how clinical researchers define and operationalize engagement in DMHI research.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study investigated how clinical researchers operationalize user engagement with DMHIs in academic settings. Investigating operationalization can help identify gaps and inform strategies to better operationalize engagement in DMHIs according to intervention goals.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n We conducted exploratory qualitative semistructured interviews via Zoom (Zoom Communications, Inc; May 2023 to February 2024) with 12 clinical mental health researchers who had developed DMHIs using human-centered design methods. We recruited participants via purposive and snowball sampling. The interviews focused on understanding what participants considered engagement, how they measured it, the strategies they used to support or increase engagement, and the barriers they faced. We inductively coded the transcripts and conducted thematic analysis, iterating on codes and themes collaboratively.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n We identified 3 dimensions of engagement for DMHIs: digital mental health components (ie, intervention, technology, and human support), levels of engagement (micro and macro), and visibility of the engagement (visible and invisible). We also described the challenges of designing DMHIs for engagement. Participants described components as overlapping; some viewed the technology and intervention as one, while others viewed the technology as distinct. Users should experience components as integrated. Within the levels of engagement, clinical researchers focused on designing for macroengagement but primarily measured microengagement through quantitative measures. Participants distinguished the visibility of engagement between what was capturable and measurable (visible engagement) and what could not be captured (invisible engagement). A major barrier to engagement was overcoming the invisibility of opportune moments for users to engage with the DMHI.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Clinical mental health research focuses on end point clinical outcomes, while user engagement refers to the dynamic interaction with DMHIs in real-world settings, encompassing behavioral, cognitive, and affective user involvement. This tension highlights the need to operationalize engagement to better understand how users engage with DMHIs. A mixed method approach to capturing engagement would better align with clinical researchers\u2019 goals of understanding macroengagement. The dimensions of engagement (components, levels, and visibility) extend previous conceptualizations by providing support at the start of designing for the engagement of DMHIs. While capturing invisible aspects can still be challenging, awareness of the visibility of engagement can help researchers align their measurement and evaluation approaches with their macroengagement goals, instead of focusing on aspects of engagement that are readily visible.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/89009","type":"journal-article","created":{"date-parts":[[2026,6,29]],"date-time":"2026-06-29T16:05:06Z","timestamp":1782749106000},"page":"e89009-e89009","source":"Crossref","is-referenced-by-count":0,"title":["Perspectives of Clinical Researchers on Engagement With Digital Mental Health Interventions: Qualitative Interview Study"],"prefix":"10.2196","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6498-2785","authenticated-orcid":false,"given":"Bruna","family":"Oewel","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-8079-6270","authenticated-orcid":false,"given":"Keertana","family":"Nambiar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8292-9232","authenticated-orcid":false,"given":"Elena","family":"Agapie","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6330-3637","authenticated-orcid":false,"given":"Madhu","family":"Reddy","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,8,6]]},"reference":[{"issue":"1","key":"R1","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2024.54718","article-title":"Prevalence of and inequities in poor mental health across 3 US surveys, 2011 to 2022","volume":"8","author":"Wright","journal-title":"JAMA Netw Open"},{"issue":"4","key":"R2","doi-asserted-by":"publisher","first-page":"394","DOI":"10.1002\/jclp.20659","article-title":"Perceived barriers to psychological treatments and their relationship to depression","volume":"66","author":"Mohr","journal-title":"J Clin Psychol"},{"key":"R3","doi-asserted-by":"publisher","DOI":"10.1145\/3544549.3574132","article-title":"Bridging HCI and implementation science for innovation adoption and public health impact","volume":"2023","author":"Lyon","journal-title":"Ext Abstr Hum Factors Computing Syst"},{"issue":"3","key":"R4","doi-asserted-by":"publisher","first-page":"325","DOI":"10.1002\/wps.20673","article-title":"The efficacy of app-supported smartphone interventions for mental health problems: a meta-analysis of randomized controlled trials","volume":"18","author":"Linardon","journal-title":"World Psychiatry"},{"issue":"1","key":"R5","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-019-0188-8","article-title":"Standalone smartphone apps for mental health: a systematic review and meta-analysis","volume":"2","author":"Weisel","journal-title":"NPJ Digit Med"},{"key":"R6","doi-asserted-by":"publisher","DOI":"10.1016\/j.brat.2023.104292","article-title":"The efficacy of self-guided internet and mobile-based interventions for preventing anxiety and depression: a systematic review and meta-analysis","volume":"164","author":"Edge","journal-title":"Behav Res Ther"},{"issue":"7","key":"R7","doi-asserted-by":"publisher","DOI":"10.2196\/12869","article-title":"Digital mental health interventions for depression, anxiety, and enhancement of psychological well-being among college students: systematic review","volume":"21","author":"Lattie","journal-title":"J Med Internet Res"},{"key":"R8","unstructured":"Comprehensive mental health action plan 2013-2030. World Health Organization; 2021. URL: https:\/\/iris.who.int\/server\/api\/core\/bitstreams\/69921758-6229-49ba-bd3d-c24736e35829\/content [Accessed 26-03-2025]"},{"issue":"3","key":"R9","doi-asserted-by":"publisher","DOI":"10.2196\/jmir.1772","article-title":"A systematic review of the impact of adherence on the effectiveness of e-therapies","volume":"13","author":"Donkin","journal-title":"J Med Internet Res"},{"issue":"5","key":"R10","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3234149","article-title":"Engagement in HCI: conception, theory and measurement","volume":"51","author":"Doherty","journal-title":"ACM Comput Surv"},{"issue":"6","key":"R11","doi-asserted-by":"publisher","DOI":"10.2196\/16730","article-title":"Engaging with a web-based psychosocial intervention for psychosis: qualitative study of user experiences","volume":"7","author":"Arnold","journal-title":"JMIR Ment Health"},{"issue":"9","key":"R12","doi-asserted-by":"publisher","first-page":"978","DOI":"10.1176\/appi.ps.201800063","article-title":"Mobile health (mHealth) versus clinic-based group intervention for people with serious mental illness: a randomized controlled trial","volume":"69","author":"Ben-Zeev","journal-title":"Psychiatr Serv"},{"issue":"7","key":"R13","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2020.10918","article-title":"Digital health and engagement-looking behind the measures and methods","volume":"3","author":"Torous","journal-title":"JAMA Netw Open"},{"issue":"7","key":"R14","doi-asserted-by":"publisher","first-page":"538","DOI":"10.1176\/appi.ps.201800519","article-title":"User engagement in mental health apps: a review of measurement, reporting, and validity","volume":"70","author":"Ng","journal-title":"Psychiatr Serv"},{"issue":"5","key":"R15","doi-asserted-by":"publisher","first-page":"863","DOI":"10.1007\/s12671-018-1050-9","article-title":"Mobile mindfulness meditation: a randomised controlled trial of the effect of two popular apps on mental health","volume":"10","author":"Flett","journal-title":"Mindfulness (N Y)"},{"issue":"9","key":"R16","doi-asserted-by":"publisher","DOI":"10.2196\/14567","article-title":"Objective user engagement with mental health apps: systematic search and panel-based usage analysis","volume":"21","author":"Baumel","journal-title":"J Med Internet Res"},{"issue":"3","key":"R17","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1007\/s40501-023-00297-3","article-title":"The engagement problem: a review of engagement with digital mental health interventions and recommendations for a path forward","volume":"10","author":"Lipschitz","journal-title":"Curr Treat Options Psychiatry"},{"issue":"2","key":"R18","doi-asserted-by":"publisher","first-page":"254","DOI":"10.1007\/s13142-016-0453-1","article-title":"Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis","volume":"7","author":"Perski","journal-title":"Transl Behav Med"},{"key":"R19","unstructured":"Bridging science and service: a report by the national advisory mental health council\u2019s clinical treatment and services research workgroup. National Institute of Mental Health (NIMH). 1999. URL: https:\/\/www.nimh.nih.gov\/about\/advisory-boards-and-groups\/namhc\/reports\/bridging-science-and-service-a-report-by-the-national-advisory-mental-health-councils-clinical-treatment-and-services-research-workgroup [Accessed 29-07-2026]"},{"issue":"2","key":"R20","doi-asserted-by":"publisher","first-page":"144","DOI":"10.1111\/j.1468-2850.2010.01204.x","article-title":"Meta-analysis of homework effects in cognitive and behavioral therapy: a replication and extension","volume":"17","author":"Kazantzis","journal-title":"Clin Psychol Sci Pract"},{"issue":"1","key":"R21","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1007\/s11020-005-1962-6","article-title":"Identifying and selecting the common elements of evidence based interventions: a distillation and matching model","volume":"7","author":"Chorpita","journal-title":"Ment Health Serv Res"},{"key":"R22","doi-asserted-by":"publisher","DOI":"10.1177\/20552076241283530","article-title":"What does it mean to be engaged with digital health interventions? A qualitative study into the experiences of engaged users and the views of professionals","volume":"10","author":"Kelders","journal-title":"Digit Health"},{"key":"R23","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2020.00926","article-title":"The concept and components of engagement in different domains applied to eHealth: a systematic scoping review","volume":"11","author":"Kelders","journal-title":"Front Psychol"},{"issue":"6","key":"R24","doi-asserted-by":"publisher","first-page":"938","DOI":"10.1002\/asi.20801","article-title":"What is user engagement? A conceptual framework for defining user engagement with technology","volume":"59","author":"O\u2019Brien","journal-title":"J Am Soc Inf Sci"},{"issue":"4","key":"R25","doi-asserted-by":"publisher","first-page":"239","DOI":"10.1177\/09637214241254328","article-title":"Towards the science of engagement with digital interventions","volume":"33","author":"Nahum-Shani","journal-title":"Curr Dir Psychol Sci"},{"issue":"5","key":"R26","doi-asserted-by":"publisher","first-page":"833","DOI":"10.1016\/j.amepre.2016.06.015","article-title":"Understanding and promoting effective engagement with digital behavior change interventions","volume":"51","author":"Yardley","journal-title":"Am J Prev Med"},{"issue":"4","key":"R27","doi-asserted-by":"publisher","DOI":"10.2196\/14052","article-title":"Understanding health behavior technology engagement: pathway to measuring digital behavior change interventions","volume":"3","author":"Cole-Lewis","journal-title":"JMIR Form Res"},{"issue":"5","key":"R28","doi-asserted-by":"publisher","DOI":"10.2196\/35048","article-title":"Engagement, predictors, and outcomes of a trauma recovery digital mental health intervention: longitudinal study","volume":"9","author":"Yeager","journal-title":"JMIR Ment Health"},{"issue":"1","key":"R29","doi-asserted-by":"publisher","DOI":"10.2196\/41428","article-title":"An integrative engagement model of digital psychotherapy: exploratory focus group findings","volume":"7","author":"Zech","journal-title":"JMIR Form Res"},{"key":"R30","unstructured":"Cho HN Eisenberg D King C Zheng K . EngageTriBoost: predictive modeling of user engagement in digital mental health intervention using explainable machine learning. arXiv. Preprint posted online on Mar 31, 2026. [doi: 10.48550\/arXiv.2604.08589]"},{"issue":"10","key":"R31","doi-asserted-by":"publisher","DOI":"10.2196\/17757","article-title":"Psychometric evaluation of the TWente Engagement with Ehealth Technologies Scale (TWEETS): evaluation study","volume":"22","author":"Kelders","journal-title":"J Med Internet Res"},{"key":"R32","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2018.01.004","article-title":"A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form","volume":"112","author":"O\u2019Brien","journal-title":"Int J Hum Comput Stud"},{"key":"R33","doi-asserted-by":"publisher","DOI":"10.1145\/3706598.3713505","article-title":"What is user engagement?: a systematic review of 241 research articles in human-computer interaction and beyond","author":"Jansen","journal-title":"Proc 2025 CHI Conf Hum Factors Comput Syst"},{"key":"R34","doi-asserted-by":"crossref","unstructured":"O\u2019Brien H . A holistic approach to measuring user engagement. In: Filimowicz M Tzankova V , editors. New Directions in Third Wave Human-Computer Interaction: Volume 2 - Methodologies. Springer; 2018:81-102. [doi: 10.14288\/1.0398527]","DOI":"10.1007\/978-3-319-73374-6_6"},{"key":"R35","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376309","article-title":"\u201cEnergy is a finite resource\u201d: designing technology to support individuals across fluctuating symptoms of depression","author":"Kornfield","journal-title":"Proc SIGCHI Conf Hum Factor Comput Syst"},{"issue":"2","key":"R36","doi-asserted-by":"publisher","DOI":"10.4108\/trans.amsys.01-06.2013.e3","article-title":"Increasing engagement with computerised cognitive behavioural therapies","volume":"1","author":"Cavanagh","journal-title":"ICST Trans Amb Sys"},{"issue":"2","key":"R37","doi-asserted-by":"publisher","first-page":"272","DOI":"10.1007\/s10608-020-10090-8","article-title":"Barriers associated with the implementation of homework in youth mental health treatment and potential mobile health solutions","volume":"45","author":"Bunnell","journal-title":"Cogn Ther Res"},{"issue":"5","key":"R38","doi-asserted-by":"publisher","DOI":"10.2196\/jmir.7725","article-title":"Accelerating digital mental health research from early design and creation to successful implementation and sustainment","volume":"19","author":"Mohr","journal-title":"J Med Internet Res"},{"issue":"2","key":"R39","doi-asserted-by":"publisher","first-page":"137","DOI":"10.1089\/tmj.2011.0062","article-title":"What design features are used in effective e-health interventions? A review using techniques from critical interpretive synthesis","volume":"18","author":"Morrison","journal-title":"Telemed J E Health"},{"issue":"4","key":"R40","doi-asserted-by":"publisher","first-page":"329","DOI":"10.1016\/j.cpr.2012.02.004","article-title":"Computer-based psychological treatments for depression: a systematic review and meta-analysis","volume":"32","author":"Richards","journal-title":"Clin Psychol Rev"},{"key":"R41","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.h5627","article-title":"Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial","volume":"351","author":"Gilbody","journal-title":"BMJ"},{"key":"R42","unstructured":"Fink A . How to Conduct Surveys: A Step-by-Step Guide. 6th ed. SAGE Publications; 2017. ISBN: 9781483378480"},{"issue":"4","key":"R43","doi-asserted-by":"publisher","first-page":"591","DOI":"10.1177\/1049732316665344","article-title":"Code saturation versus meaning saturation: how many interviews are enough?","volume":"27","author":"Hennink","journal-title":"Qual Health Res"},{"issue":"2","key":"R44","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1191\/1478088706qp063oa","article-title":"Using thematic analysis in psychology","volume":"3","author":"Braun","journal-title":"Qual Res Psychol"},{"key":"R45","unstructured":"Salda\u00f1a J . The Coding Manual for Qualitative Researchers. 5th ed. SAGE Publications; 2025. ISBN: 9781036202187"},{"issue":"4","key":"R46","doi-asserted-by":"publisher","first-page":"517","DOI":"10.1177\/0011000097254001","article-title":"A guide to conducting consensual qualitative research","volume":"25","author":"Hill","journal-title":"Couns Psychol"},{"issue":"1","key":"R47","doi-asserted-by":"publisher","first-page":"1609406918786362","DOI":"10.1177\/1609406918786362","article-title":"Ensuring rigor in qualitative data analysis: a design research approach to coding combining NVivo with traditional material methods","volume":"17","author":"Maher","journal-title":"Int J Qual Methods"},{"issue":"8","key":"R48","doi-asserted-by":"publisher","first-page":"1093","DOI":"10.1037\/amp0000654","article-title":"Scaling evidence-based treatments through digital mental health","volume":"75","author":"Schueller","journal-title":"Am Psychol"},{"issue":"1","key":"R49","doi-asserted-by":"publisher","DOI":"10.2196\/jmir.1602","article-title":"Supportive accountability: a model for providing human support to enhance adherence to eHealth interventions","volume":"13","author":"Mohr","journal-title":"J Med Internet Res"},{"key":"R50","doi-asserted-by":"publisher","DOI":"10.1177\/20552076231225591","article-title":"Macro-engagement in mHealth: exploring user engagement beyond the screen","volume":"10","author":"Villegas Mej\u00eda","journal-title":"Digit Health"},{"issue":"5","key":"R51","doi-asserted-by":"publisher","DOI":"10.1192\/bjo.2020.72","article-title":"Beyond clicks and downloads: a call for a more comprehensive approach to measuring mobile-health app engagement","volume":"6","author":"O\u2019Brien","journal-title":"BJPsych Open"},{"key":"R52","doi-asserted-by":"publisher","DOI":"10.1016\/j.invent.2021.100403","article-title":"Targeting subjective engagement in experimental therapeutics for digital mental health interventions","volume":"25","author":"Graham","journal-title":"Internet Interv"},{"key":"R53","doi-asserted-by":"publisher","DOI":"10.1145\/3706598.3713143","article-title":"Understanding mental wellbeing and tools for support with Taiwanese emerging adults: an eastern cultural perspective","author":"Cheng","journal-title":"Proc 2025 CHI Conf Hum Factors Comput Syst"},{"key":"R54","doi-asserted-by":"publisher","DOI":"10.1145\/3706598.3714142","article-title":"Healing through stories: co-designing digital mental health with Asian Americans","author":"Bautista","journal-title":"Proc 2025 CHI Conf Hum Factors Comput Syst New York"},{"issue":"3","key":"R55","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1093\/iwcomp\/iwaa019","article-title":"The design of ecological momentary assessment technologies","volume":"32","author":"Doherty","journal-title":"Interact Comput"},{"key":"R56","doi-asserted-by":"publisher","DOI":"10.1145\/3706598.3714422","article-title":"Micro-phenomenology as a method for studying user experience in human-computer interaction","author":"Heimann","journal-title":"Proc 2025 CHI Conf Hum Factors Comput Syst"},{"key":"R57","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2023.104359","article-title":"AwarNS: a framework for developing context-aware reactive mobile applications for health and mental health","volume":"141","author":"Gonz\u00e1lez-P\u00e9rez","journal-title":"J Biomed Inform"}],"container-title":["JMIR Formative Research"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,8,6]],"date-time":"2026-08-06T16:42:05Z","timestamp":1786034525000},"score":1,"resource":{"primary":{"URL":"https:\/\/formative.jmir.org\/2026\/1\/e89009"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,6]]},"references-count":57,"URL":"https:\/\/doi.org\/10.2196\/89009","relation":{},"ISSN":["2561-326X"],"issn-type":[{"value":"2561-326X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,6]]},"article-number":"v10i1e89009"}}