{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T14:31:18Z","timestamp":1780410678721,"version":"3.54.1"},"reference-count":53,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2021,2,28]],"date-time":"2021-02-28T00:00:00Z","timestamp":1614470400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100000133","name":"Agency for Healthcare Research and Quality","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100000133","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,6,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Objective<\/jats:title><jats:p>Providing behavioral health interventions via smartphones allows these interventions to be adapted to the changing behavior, preferences, and needs of individuals. This can be achieved through reinforcement learning (RL), a sub-area of machine learning. However, many challenges could affect the effectiveness of these algorithms in the real world. We provide guidelines for decision-making.<\/jats:p><\/jats:sec><jats:sec><jats:title>Materials and Methods<\/jats:title><jats:p>Using thematic analysis, we describe challenges, considerations, and solutions for algorithm design decisions in a collaboration between health services researchers, clinicians, and data scientists. We use the design process of an RL algorithm for a mobile health study \u201cDIAMANTE\u201d for increasing physical activity in underserved patients with diabetes and depression. Over the 1.5-year project, we kept track of the research process using collaborative cloud Google Documents, Whatsapp messenger, and video teleconferencing. We discussed, categorized, and coded critical challenges. We grouped challenges to create thematic topic process domains.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Nine challenges emerged, which we divided into 3 major themes: 1. Choosing the model for decision-making, including appropriate contextual and reward variables; 2. Data handling\/collection, such as how to deal with missing or incorrect data in real-time; 3. Weighing the algorithm performance vs effectiveness\/implementation in real-world settings.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>The creation of effective behavioral health interventions does not depend only on final algorithm performance. Many decisions in the real world are necessary to formulate the design of problem parameters to which an algorithm is applied. Researchers must document and evaulate these considerations and decisions before and during the intervention period, to increase transparency, accountability, and reproducibility.<\/jats:p><\/jats:sec><jats:sec><jats:title>Trial Registration<\/jats:title><jats:p>clinicaltrials.gov, NCT03490253.<\/jats:p><\/jats:sec>","DOI":"10.1093\/jamia\/ocab001","type":"journal-article","created":{"date-parts":[[2021,1,7]],"date-time":"2021-01-07T15:42:22Z","timestamp":1610034142000},"page":"1225-1234","source":"Crossref","is-referenced-by-count":30,"title":["Adaptive learning algorithms to optimize mobile applications for behavioral health: guidelines for design decisions"],"prefix":"10.1093","volume":"28","author":[{"given":"Caroline A","family":"Figueroa","sequence":"first","affiliation":[{"name":"School of Social Welfare, University of California Berkeley, Berkeley, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Adrian","family":"Aguilera","sequence":"additional","affiliation":[{"name":"School of Social Welfare, University of California Berkeley, Berkeley, California, USA"},{"name":"UCSF Center for Vulnerable Populations, Zuckerberg San Francisco General Hospital, San Francisco, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bibhas","family":"Chakraborty","sequence":"additional","affiliation":[{"name":"Centre for Quantitative Medicine, Duke-National University of Singapore Medical School, Singapore, Singapore"},{"name":"Department of Statistics and Applied Probability, National University of Singapore, Singapore, Singapore"},{"name":"Department of Biostatistics and Bioinformatics, Duke University, Durham, North Carolina, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Arghavan","family":"Modiri","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Toronto, Toronto, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jai","family":"Aggarwal","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Toronto, Toronto, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nina","family":"Deliu","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Toronto, Toronto, Canada"},{"name":"Department of Statistical Sciences, Sapienza University of Rome, Rome, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Urmimala","family":"Sarkar","sequence":"additional","affiliation":[{"name":"UCSF Center for Vulnerable Populations, Zuckerberg San Francisco General Hospital, San Francisco, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joseph","family":"Jay Williams","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Toronto, Toronto, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1111-8595","authenticated-orcid":false,"given":"Courtney R","family":"Lyles","sequence":"additional","affiliation":[{"name":"UCSF Center for Vulnerable Populations, Zuckerberg San Francisco General Hospital, San Francisco, California, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2021,2,28]]},"reference":[{"key":"2021061318591482200_ocab001-B1","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.jad.2017.04.046","article-title":"Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials","volume":"218","author":"Firth","year":"2017","journal-title":"J Affect Disord"},{"issue":"3","key":"2021061318591482200_ocab001-B2","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1002\/wps.20472","article-title":"The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials","volume":"16","author":"Firth","year":"2017","journal-title":"World Psychiatry"},{"key":"2021061318591482200_ocab001-B3","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1016\/j.ypmed.2017.08.022","article-title":"Effectiveness of mobile health (mHealth) interventions for promoting healthy eating in adults: a systematic review","volume":"105","author":"McCarroll","year":"2017","journal-title":"Prevent Med"},{"issue":"1","key":"2021061318591482200_ocab001-B4","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1186\/s40798-018-0157-9","article-title":"The effect of physical activity interventions comprising wearables and smartphone applications on physical activity: a systematic review and meta-analysis","volume":"4","author":"Gal","year":"2018","journal-title":"Sports Med Open"},{"key":"2021061318591482200_ocab001-B5","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1016\/j.socscimed.2017.09.021","article-title":"Effectiveness of physical activity interventions in achieving behaviour change maintenance in young and middle aged adults: a systematic review and meta-analysis","volume":"192","author":"Murray","year":"2017","journal-title":"Soc Sci Med"},{"issue":"6","key":"2021061318591482200_ocab001-B6","doi-asserted-by":"crossref","first-page":"704","DOI":"10.1007\/s11764-017-0632-1","article-title":"Digital health behaviour change interventions targeting physical activity and diet in cancer survivors: a systematic review and meta-analysis","volume":"11","author":"Roberts","year":"2017","journal-title":"J Cancer Surviv"},{"issue":"9","key":"2021061318591482200_ocab001-B7","doi-asserted-by":"crossref","first-page":"e14567","DOI":"10.2196\/14567","article-title":"Objective user engagement with mental health apps: systematic search and panel-based usage analysis","volume":"21","author":"Baumel","year":"2019","journal-title":"J Med Internet Res"},{"issue":"3","key":"2021061318591482200_ocab001-B8","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1177\/1062860618798469","article-title":"Innovative implementation studies conducted in US Safety Net Health Care Settings: a systematic review","volume":"34","author":"Lyles","year":"2019","journal-title":"Am J Med Qual"},{"issue":"1","key":"2021061318591482200_ocab001-B9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41746-019-0206-x","article-title":"What is the clinical value of mHealth for patients?","volume":"3","author":"Rowland","year":"2020","journal-title":"NPJ Digit Med"},{"issue":"1","key":"2021061318591482200_ocab001-B10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41746-019-0188-8","article-title":"Standalone smartphone apps for mental health\u2014a systematic review and meta-analysis","volume":"2","author":"Weisel","year":"2019","journal-title":"NPJ Digit Med"},{"issue":"6","key":"2021061318591482200_ocab001-B11","doi-asserted-by":"crossref","first-page":"446","DOI":"10.1007\/s12160-016-9830-8","article-title":"Just-in-Time Adaptive Interventions (JITAIs) in mobile health: key components and design principles for ongoing health behavior support","volume":"52","author":"Nahum-Shani","year":"2018","journal-title":"Ann Behav Med"},{"issue":"4","key":"2021061318591482200_ocab001-B12","doi-asserted-by":"crossref","first-page":"e12286","DOI":"10.2196\/12286","article-title":"Applications of machine learning in real-life digital health interventions: review of the literature","volume":"21","author":"Triantafyllidis","year":"2019","journal-title":"J Med Internet Res"},{"key":"2021061318591482200_ocab001-B13","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1007\/978-3-030-31620-4_18","volume-title":"Digital Phenotyping and Mobile Sensing","author":"Rabbi","year":"2019"},{"key":"2021061318591482200_ocab001-B14","author":"Yu","year":"2019"},{"issue":"1","key":"2021061318591482200_ocab001-B15","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1007\/s12160-014-9634-7","article-title":"The potential impact of intelligent systems for mobile health self-management support: Monte Carlo simulations of text message support for medication adherence","volume":"49","author":"Piette","year":"2015","journal-title":"Ann Behav Med"},{"issue":"1","key":"2021061318591482200_ocab001-B16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3381007","article-title":"Personalized heartsteps: a reinforcement learning algorithm for optimizing physical activity","volume":"4","author":"Liao","year":"2020","journal-title":"Proc ACM Interact Mob Wearable Ubiquitous Technol"},{"issue":"10","key":"2021061318591482200_ocab001-B17","doi-asserted-by":"crossref","first-page":"e338","DOI":"10.2196\/jmir.7994","article-title":"Encouraging physical activity in patients with diabetes: intervention using a reinforcement learning system","volume":"19","author":"Yom-Tov","year":"2017","journal-title":"J Med Internet Res"},{"issue":"7126","key":"2021061318591482200_ocab001-B18","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1136\/bmj.316.7126.201","article-title":"Understanding controlled trials. Why are randomised controlled trials important?","volume":"316","author":"Sibbald","year":"1998","journal-title":"BMJ"},{"issue":"1","key":"2021061318591482200_ocab001-B19","doi-asserted-by":"crossref","first-page":"88","DOI":"10.1007\/s00267-013-0115-8","article-title":"Challenges of interdisciplinary research: reconciling qualitative and quantitative methods for understanding human\u2013landscape systems","volume":"53","author":"Lach","year":"2014","journal-title":"Environ Manag"},{"issue":"8","key":"2021061318591482200_ocab001-B20","doi-asserted-by":"crossref","first-page":"e034723","DOI":"10.1136\/bmjopen-2019-034723","article-title":"mHealth app using machine learning to increase physical activity in diabetes and depression: clinical trial protocol for the DIAMANTE Study","volume":"10","author":"Aguilera","year":"2020","journal-title":"BMJ Open"},{"issue":"1","key":"2021061318591482200_ocab001-B21","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1186\/1748-5908-6-42","article-title":"The behaviour change wheel: a new method for characterising and designing behaviour change interventions","volume":"6","author":"Michie","year":"2011","journal-title":"Implement Sci"},{"key":"2021061318591482200_ocab001-B22","first-page":"21177","article-title":"Combining user centered design and crowdsourcing to develop messaging content for a physical activity smartphone application tailored to low-income patients","author":"G\u00f3mez-Pathak","year":"2020","journal-title":"JMIR"},{"issue":"Suppl","key":"2021061318591482200_ocab001-B23","doi-asserted-by":"crossref","first-page":"1220","DOI":"10.1037\/hea0000305","article-title":"Microrandomized trials: an experimental design for developing just-in-time adaptive interventions","volume":"34","author":"Klasnja","year":"2015","journal-title":"Health Psychol"},{"key":"2021061318591482200_ocab001-B24","first-page":"167","article-title":"Recent advances in multiarmed bandits for sequential decision making","author":"Agrawal","year":"2019","journal-title":"informsPubsOnline"},{"key":"2021061318591482200_ocab001-B25","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1007\/978-3-319-51394-2_25","volume-title":"Mobile Health","author":"Tewari","year":"2017"},{"key":"2021061318591482200_ocab001-B26","year":"2017"},{"key":"2021061318591482200_ocab001-B27","year":"16, 2012: 1\u201339."},{"key":"2021061318591482200_ocab001-B28"},{"key":"2021061318591482200_ocab001-B29","first-page":"707","year":"2015"},{"key":"2021061318591482200_ocab001-B30","author":"Tomkins","year":"2020."},{"key":"2021061318591482200_ocab001-B31","first-page":"1","year":"2020"},{"key":"2021061318591482200_ocab001-B32","year":"2019"},{"issue":"1","key":"2021061318591482200_ocab001-B33","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1080\/00031305.1975.10479105","article-title":"Ridge regression in practice","volume":"29","author":"Marquardt","year":"1975","journal-title":"Am Statist"},{"issue":"482","key":"2021061318591482200_ocab001-B34","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1198\/016214508000000337","article-title":"The Bayesian lasso","volume":"103","author":"Park","year":"2008","journal-title":"J Am Statist Assoc"},{"issue":"11","key":"2021061318591482200_ocab001-B35","doi-asserted-by":"crossref","first-page":"e0206828","DOI":"10.1371\/journal.pone.0206828","article-title":"Assessing a novel way to measure step count while walking using a custom mobile phone application","volume":"13","author":"Hurt","year":"2018","journal-title":"PLoS One"},{"key":"2021061318591482200_ocab001-B36","doi-asserted-by":"crossref","DOI":"10.1176\/appi.ajp.2009.09040458","article-title":"Last observation carried forward versus mixed models in the analysis of psychiatric clinical trials","author":"Hamer","year":"2009","journal-title":"Am Psychiatric Assoc"},{"issue":"5","key":"2021061318591482200_ocab001-B37","doi-asserted-by":"crossref","first-page":"e54","DOI":"10.2196\/jmir.1448","article-title":"Missing data approaches in eHealth research: simulation study and a tutorial for nonmathematically inclined researchers","volume":"12","author":"Blankers","year":"2010","journal-title":"J Med Internet Res"},{"key":"2021061318591482200_ocab001-B38","author":"Russo","year":"2017"},{"issue":"4","key":"2021061318591482200_ocab001-B39","doi-asserted-by":"crossref","first-page":"S346","DOI":"10.1016\/j.amepre.2012.12.009","article-title":"Preventing type 2 diabetes in communities across the US: the National Diabetes Prevention Program","volume":"44","author":"Albright","year":"2013","journal-title":"Am J Prevent Med"},{"issue":"3","key":"2021061318591482200_ocab001-B40","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1097\/MLR.0b013e3182408812","article-title":"Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact","volume":"50","author":"Curran","year":"2012","journal-title":"Med Care"},{"issue":"5","key":"2021061318591482200_ocab001-B41","doi-asserted-by":"crossref","first-page":"e153","DOI":"10.2196\/jmir.7725","article-title":"Accelerating digital mental health research from early design and creation to successful implementation and sustainment","volume":"19","author":"Mohr","year":"2017","journal-title":"J Med Internet Res"},{"issue":"1","key":"2021061318591482200_ocab001-B42","doi-asserted-by":"crossref","first-page":"e11","DOI":"10.2196\/jmir.7.1.e11","article-title":"The law of attrition","volume":"7","author":"Eysenbach","year":"2005","journal-title":"J Med Internet Res"},{"key":"2021061318591482200_ocab001-B43","first-page":"39","article-title":"Who benefits most from adding technology to depression treatment and how? An analysis of engagement with a texting adjunct for psychotherapy","author":"Figueroa"},{"issue":"4","key":"2021061318591482200_ocab001-B44","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1093\/jamiaopen\/ooz049","article-title":"Engaging users in the design of an mHealth, text message-based intervention to increase physical activity at a safety-net health care system","volume":"2","author":"Avila-Garcia","year":"2019","journal-title":"JAMIA Open"},{"issue":"8","key":"2021061318591482200_ocab001-B45","doi-asserted-by":"crossref","first-page":"e14250","DOI":"10.2196\/14250","article-title":"Assessing mobile phone digital literacy and engagement in user-centered design in a diverse, safety-net population: mixed methods study","volume":"7","author":"Nouri","year":"2019","journal-title":"JMIR Mhealth Uhealth"},{"key":"2021061318591482200_ocab001-B46","article-title":"Machine learning and artificial intelligence research for patient benefit: 20 critical questions on transparency, replicability, ethics, and effectiveness","volume":"368","author":"Vollmer","year":"2020","journal-title":"BMJ"},{"issue":"1","key":"2021061318591482200_ocab001-B47","first-page":"1","article-title":"The reproducibility crisis in the age of digital medicine","volume":"2","author":"Stupple","year":"2019","journal-title":"NPJ Digit Med"},{"key":"2021061318591482200_ocab001-B48","doi-asserted-by":"crossref","DOI":"10.1136\/bmj.m3164","article-title":"Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI Extension","author":"Liu","year":"2020","journal-title":"BMJ"},{"issue":"5","key":"2021061318591482200_ocab001-B49","doi-asserted-by":"crossref","first-page":"e99","DOI":"10.2196\/jmir.5357","article-title":"The emergence of personalized health technology","volume":"18","author":"Allen","year":"2016","journal-title":"J Med Internet Res"},{"issue":"3","key":"2021061318591482200_ocab001-B50","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1007\/s40501-019-00181-z","article-title":"Use of digital mental health for marginalized and underserved populations","volume":"6","author":"Schueller","year":"2019","journal-title":"Curr Treat Options Psych"},{"issue":"11","key":"2021061318591482200_ocab001-B51","doi-asserted-by":"crossref","first-page":"e292","DOI":"10.2196\/jmir.9397","article-title":"Measuring engagement in eHealth and mHealth behavior change interventions: viewpoint of methodologies","volume":"20","author":"Short","year":"2018","journal-title":"J Med Internet Res"},{"issue":"7","key":"2021061318591482200_ocab001-B52","doi-asserted-by":"crossref","first-page":"1818","DOI":"10.1111\/sms.13074","article-title":"Validity of activity trackers, smartphones, and phone applications to measure steps in various walking conditions","volume":"28","author":"H\u00f6chsmann","year":"2018","journal-title":"Scand J Med Sci Sports"},{"issue":"2","key":"2021061318591482200_ocab001-B53","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1016\/j.jesf.2018.05.001","article-title":"Accuracy of a smartphone pedometer application according to different speeds and mobile phone locations in a laboratory context","volume":"16","author":"Presset","year":"2018","journal-title":"J Exerc Sci Fit"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/6\/1225\/38615357\/ocab001.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/6\/1225\/38615357\/ocab001.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,22]],"date-time":"2024-08-22T01:22:32Z","timestamp":1724289752000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/28\/6\/1225\/6154382"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,28]]},"references-count":53,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2021,2,28]]},"published-print":{"date-parts":[[2021,6,12]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocab001","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,6,1]]},"published":{"date-parts":[[2021,2,28]]}}}