{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,21]],"date-time":"2026-08-21T12:17:16Z","timestamp":1787314636181,"version":"build-2736575974"},"reference-count":147,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,9,25]],"date-time":"2023-09-25T00:00:00Z","timestamp":1695600000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Psychiatry"],"abstract":"<jats:sec>\n                    <jats:title>Objectives<\/jats:title>\n                    <jats:p>Anxiety and mood disorders greatly affect the quality of life for individuals worldwide. A substantial proportion of patients do not sufficiently improve during evidence-based treatments in mental healthcare. It remains challenging to predict which patients will or will not benefit. Moreover, the limited research available on predictors of treatment outcomes comes from efficacy RCTs with strict selection criteria which may limit generalizability to a real-world context. The current study evaluates the performance of different machine learning (ML) models in predicting non-improvement in an observational sample of patients treated in routine specialized mental healthcare.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>In the current longitudinal exploratory prediction study diagnosis-related, sociodemographic, clinical and routinely collected patient-reported quantitative outcome measures were acquired during treatment as usual of 755 patients with a primary anxiety, depressive, obsessive compulsive or trauma-related disorder in a specialized outpatient mental healthcare center. ML algorithms were trained to predict non-response (&amp;lt; 0.5 standard deviation improvement) in symptomatic distress 6\u2009months after baseline. Different models were trained, including models with and without early change scores in psychopathology and well-being and models with a trimmed set of predictor variables. Performance of trained models was evaluated in a hold-out sample (30%) as a proxy for unseen data.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>ML models without early change scores performed poorly in predicting six-month non-response in the hold-out sample with Area Under the Curves (AUCs) &amp;lt; 0.63. Including early change scores slightly improved the models\u2019 performance (AUC range: 0.68\u20130.73). Computationally-intensive ML models did not significantly outperform logistic regression (AUC: 0.69). Reduced prediction models performed similar to the full prediction models in both the models without (AUC: 0.58\u20130.62 vs. 0.58\u20130.63) and models with early change scores (AUC: 0.69\u20130.73 vs. 0.68\u20130.71). Across different ML algorithms, early change scores in psychopathology and well-being consistently emerged as important predictors for non-improvement.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Accurately predicting treatment outcomes in a mental healthcare context remains challenging. While advanced ML algorithms offer flexibility, they showed limited additional value compared to traditional logistic regression in this study. The current study confirmed the importance of taking early change scores in both psychopathology and well-being into account for predicting longer-term outcomes in symptomatic distress.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fpsyt.2023.1236551","type":"journal-article","created":{"date-parts":[[2023,9,26]],"date-time":"2023-09-26T00:44:56Z","timestamp":1695689096000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Predicting non-improvement of symptoms in daily mental healthcare practice using routinely collected patient-level data: a machine learning approach"],"prefix":"10.3389","volume":"14","author":[{"given":"Katinka","family":"Franken","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peter","family":"ten Klooster","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ernst","family":"Bohlmeijer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gerben","family":"Westerhof","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jannis","family":"Kraiss","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,9,25]]},"reference":[{"key":"ref1","volume-title":"World mental health report: Transforming mental health for all","year":"2022"},{"key":"ref2","doi-asserted-by":"publisher","first-page":"e0116820","DOI":"10.1371\/journal.pone.0116820","article-title":"The global burden of mental, neurological and substance use disorders: an analysis from the global burden of disease study 2010","volume":"10","author":"Whiteford","year":"2015","journal-title":"PLoS One"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"603","DOI":"10.1016\/S2215-0366(16)30131-6","article-title":"Scaling-up of treatment of depression and anxiety\u2013Authors\u2019 reply","volume":"3","author":"Chisholm","year":"2016","journal-title":"Lancet Psychiatry"},{"key":"ref4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12889-023-15382-8","article-title":"A needs assessment for self-management services for adults awaiting community-based mental health services","volume":"23","author":"Jennings","year":"2023","journal-title":"BMC Public Health"},{"key":"ref5","doi-asserted-by":"publisher","first-page":"541","DOI":"10.1002\/da.22755","article-title":"Individual treatment selection for patients with posttraumatic stress disorder","volume":"35","author":"Deisenhofer","year":"2018","journal-title":"Depress Anxiety"},{"key":"ref6","doi-asserted-by":"publisher","first-page":"835","DOI":"10.1037\/ccp0000231","article-title":"Case complexity as a guide for psychological treatment selection","volume":"85","author":"Delgadillo","year":"2017","journal-title":"J Consult Clin Psychol"},{"key":"ref7","doi-asserted-by":"publisher","first-page":"159","DOI":"10.1037\/0022-006X.69.2.159","article-title":"Patient-focused research: using patient outcome data to enhance treatment effects","volume":"69","author":"Lambert","year":"2001","journal-title":"J Consult Clin Psychol"},{"key":"ref8","doi-asserted-by":"publisher","first-page":"341","DOI":"10.1177\/0011000005285875","article-title":"The meta-analysis of clinical judgment project: fifty-six years of accumulated research on clinical versus statistical prediction","volume":"34","author":"\u00c6gisd\u00f3ttir","year":"2006","journal-title":"Couns Psychol"},{"key":"ref9","volume-title":"Treatment-resistant mood disorders","author":"Carvalho","year":"2015"},{"key":"ref10","doi-asserted-by":"publisher","first-page":"118","DOI":"10.1016\/j.jad.2014.02.026","article-title":"The effects of psychotherapies for major depression in adults on remission, recovery and improvement: a meta-analysis","volume":"159","author":"Cuijpers","year":"2014","journal-title":"J Affect Disord"},{"key":"ref11","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s40337-017-0164-0","article-title":"Identifying fundamental criteria for eating disorder recovery: a systematic review and qualitative meta-analysis","volume":"5","author":"De Vos","year":"2017","journal-title":"J Eat Disord"},{"key":"ref12","doi-asserted-by":"publisher","first-page":"1041","DOI":"10.1176\/appi.ajp.2013.12070899","article-title":"The efficacy of cognitive-behavioral therapy and psychodynamic therapy in the outpatient treatment of major depression: a randomized clinical trial","volume":"170","author":"Driessen","year":"2013","journal-title":"Am J Psychiatr"},{"key":"ref13","doi-asserted-by":"publisher","first-page":"e1001454","DOI":"10.1371\/journal.pmed.1001454","article-title":"Comparative efficacy of seven psychotherapeutic interventions for patients with depression: a network meta-analysis","volume":"10","author":"Barth","year":"2013","journal-title":"PLoSMedicine"},{"key":"ref14","doi-asserted-by":"publisher","first-page":"245","DOI":"10.1002\/wps.20346","article-title":"How effective are cognitive behavior therapies for major depression and anxiety disorders? A meta-analytic update of the evidence","volume":"15","author":"Cuijpers","year":"2016","journal-title":"World Psychiatry"},{"key":"ref15","doi-asserted-by":"publisher","first-page":"e1316","DOI":"10.6061\/clinics\/2019\/e1316","article-title":"Treatment of anxiety disorders in clinical practice: a critical overview of recent systematic evidence","volume":"74","author":"Mangolini","year":"2019","journal-title":"Clinics"},{"key":"ref16","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1037\/pst0000193","article-title":"Psychotherapy relationships that work III","volume":"55","author":"Norcross","year":"2018","journal-title":"Psychotherapy"},{"key":"ref17","doi-asserted-by":"publisher","first-page":"612","DOI":"10.1177\/070674371305801106","article-title":"Overdiagnosis problems in the DSM-IV and the new DSM-5: can they be resolved by the distress\u2013impairment criterion?","volume":"58","author":"Bolton","year":"2013","journal-title":"Can J Psychiatry"},{"key":"ref18","doi-asserted-by":"publisher","first-page":"223","DOI":"10.1016\/j.bpsc.2017.11.007","article-title":"Machine learning for precision psychiatry: opportunities and challenges","volume":"3","author":"Bzdok","year":"2018","journal-title":"Biol Psychiatry"},{"key":"ref19","doi-asserted-by":"publisher","first-page":"52","DOI":"10.1080\/10503307.2020.1839140","article-title":"Identifying CBT non-response among OCD outpatients: a machine-learning approach","volume":"31","author":"Hilbert","year":"2021","journal-title":"Psychother Res"},{"key":"ref20","doi-asserted-by":"publisher","first-page":"76","DOI":"10.1016\/j.brat.2015.06.002","article-title":"Stepped care versus face-to\u2013face cognitive behavior therapy for panic disorder and social anxiety disorder: predictors and moderators of outcome","volume":"71","author":"Haug","year":"2015","journal-title":"Behav Res Ther"},{"key":"ref21","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1192\/bjpo.bp.115.000745","article-title":"Patient characteristics as a moderator of posttraumatic stress disorder treatment outcome: combining symptom burden and strengths","volume":"2","author":"Cloitre","year":"2016","journal-title":"BJPsych open"},{"key":"ref22","doi-asserted-by":"publisher","first-page":"90","DOI":"10.1037\/a0038878","article-title":"Examining patient characteristics as predictors of patient beliefs about treatment credibility and expectancies for treatment outcome","volume":"25","author":"Cohen","year":"2015","journal-title":"J Psychother Integr"},{"key":"ref23","doi-asserted-by":"publisher","first-page":"706","DOI":"10.1037\/cou0000424","article-title":"Assessing the alliance\u2013outcome association adjusted for patient characteristics and treatment processes: a meta-analytic summary of direct comparisons","volume":"67","author":"Fl\u00fcckiger","year":"2020","journal-title":"J Couns Psychol"},{"key":"ref24","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.brat.2015.07.006","article-title":"The impact of comorbid personality difficulties on response to IAPT treatment for depression and anxiety","volume":"73","author":"Goddard","year":"2015","journal-title":"Behav Res Ther"},{"key":"ref25","doi-asserted-by":"publisher","first-page":"484","DOI":"10.1016\/j.psychres.2018.11.068","article-title":"Pre-treatment patient characteristics as predictors of drop-out and treatment outcome in individual and family therapy for adolescents and adults with anorexia nervosa: a systematic review and meta-analysis","volume":"271","author":"Gregertsen","year":"2019","journal-title":"Psychiatry Res"},{"key":"ref26","doi-asserted-by":"publisher","first-page":"875","DOI":"10.1016\/S0272-7358(02)00106-X","article-title":"Cognitive therapy of depression: pretreatment patient predictors of outcome","volume":"22","author":"Hamilton","year":"2002","journal-title":"Clin Psychol Rev"},{"key":"ref27","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1002\/cpp.1936","article-title":"Baseline patient characteristics predicting outcome and attrition in cognitive therapy for social phobia: results from a large multicentre trial","volume":"23","author":"Hoyer","year":"2016","journal-title":"Clin Psychol Psychother"},{"key":"ref28","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1017\/S2045796016000020","article-title":"Using patient self-reports to study heterogeneity of treatment effects in major depressive disorder","volume":"26","author":"Kessler","year":"2017","journal-title":"Epidemiol Psychiatr Sci"},{"key":"ref29","doi-asserted-by":"publisher","first-page":"1067","DOI":"10.1016\/j.cpr.2013.08.008","article-title":"A systematic review of predictors and moderators of response to psychological therapies in OCD: do we have enough empirical evidence to target treatment?","volume":"33","author":"Knopp","year":"2013","journal-title":"Clin Psychol Rev"},{"key":"ref30","doi-asserted-by":"publisher","first-page":"223","DOI":"10.1002\/cpp.496","article-title":"The probability of treatment success, failure and duration\u2014what can be learned from empirical data to support decision making in clinical practice?","volume":"13","author":"Lutz","year":"2006","journal-title":"Clin Psychol Psychother"},{"key":"ref31","doi-asserted-by":"publisher","first-page":"92","DOI":"10.1590\/S1516-44462012000100016","article-title":"A review on predictors of treatment outcome in social anxiety disorder","volume":"34","author":"Mululo","year":"2012","journal-title":"Braz J Psychiatry"},{"key":"ref32","doi-asserted-by":"publisher","first-page":"455","DOI":"10.1080\/16506073.2019.1669701","article-title":"Predictors of outcome in guided self-help cognitive behavioural therapy for common mental disorders in primary care","volume":"49","author":"Salomonsson","year":"2020","journal-title":"Cogn Behav Ther"},{"key":"ref33","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1016\/j.genhosppsych.2022.03.001","article-title":"Using clinical patient characteristics to predict treatment outcome of cognitive behavior therapies for individuals with medically unexplained symptoms: a systematic review and meta-analysis","volume":"77","author":"Sarter","year":"2022","journal-title":"Gen Hosp Psychiatry"},{"key":"ref34","doi-asserted-by":"publisher","first-page":"94","DOI":"10.3109\/08039480903426929","article-title":"Pre-treatment patient variables as predictors of drop-out and treatment outcome in cognitive behavioural therapy for social phobia: a systematic review","volume":"64","author":"Eskildsen","year":"2010","journal-title":"Nord J Psychiatry"},{"key":"ref35","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1080\/j.1440-1614.2006.01736.x","article-title":"Cognitive behavioural therapy for depression, panic disorder and generalized anxiety disorder: a meta-regression of factors that may predict outcome","volume":"40","author":"Haby","year":"2006","journal-title":"Aust N Z J Psychiatry"},{"key":"ref36","doi-asserted-by":"publisher","first-page":"94","DOI":"10.1037\/0021-9010.90.1.94","article-title":"Effect size and power in assessing moderating effects of categorical variables using multiple regression: a 30-year review","volume":"90","author":"Aguinis","year":"2005","journal-title":"J Appl Psychol"},{"key":"ref37","doi-asserted-by":"publisher","first-page":"445","DOI":"10.1177\/2167702618815466","article-title":"Sample size requirements for multivariate models to predict between-patient differences in best treatments of major depressive disorder","volume":"7","author":"Luedtke","year":"2019","journal-title":"Clin Psychol Sci"},{"key":"ref38","doi-asserted-by":"publisher","first-page":"108","DOI":"10.1016\/j.msksp.2017.10.002","article-title":"Sample size estimation for cluster randomized controlled trials","volume":"34","author":"Ribeiro","year":"2018","journal-title":"Musculoskeletal Sci Pract"},{"key":"ref39","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s13063-018-2886-y","article-title":"A study of target effect sizes in randomised controlled trials published in the health technology assessment journal","volume":"19","author":"Rothwell","year":"2018","journal-title":"Trials"},{"key":"ref40","doi-asserted-by":"publisher","first-page":"103450","DOI":"10.1016\/j.ijnurstu.2019.103450","article-title":"Reporting sample size calculations for randomized controlled trials published in nursing journals: a cross-sectional study","volume":"102","author":"Tam","year":"2020","journal-title":"Int J Nurs Stud"},{"key":"ref41","doi-asserted-by":"publisher","first-page":"1057","DOI":"10.1177\/0962280215588241","article-title":"Estimating the sample size for a pilot randomised trial to minimise the overall trial sample size for the external pilot and main trial for a continuous outcome variable","volume":"25","author":"Whitehead","year":"2016","journal-title":"Stat Methods Med Res"},{"key":"ref42","doi-asserted-by":"publisher","first-page":"2175","DOI":"10.1056\/NEJMms1604593","article-title":"Assessing the gold standard\u2013lessons from the history of RCTs","volume":"374","author":"Bothwell","year":"2016","journal-title":"Mass Medical Soc"},{"key":"ref43","doi-asserted-by":"publisher","first-page":"465","DOI":"10.1056\/NEJMra1614394","article-title":"Evidence for health decision making\u2014beyond randomized, controlled trials","volume":"377","author":"Frieden","year":"2017","journal-title":"N Engl J Med"},{"key":"ref44","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1016\/S0140-6736(04)17670-8","article-title":"External validity of randomised controlled trials:\u201cto whom do the results of this trial apply?\u201d","volume":"365","author":"Rothwell","year":"2005","journal-title":"Lancet"},{"key":"ref45","doi-asserted-by":"publisher","first-page":"475","DOI":"10.1007\/s11121-014-0513-z","article-title":"Assessing the generalizability of randomized trial results to target populations","volume":"16","author":"Stuart","year":"2015","journal-title":"Prev Sci"},{"key":"ref46","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1016\/j.jad.2016.03.011","article-title":"Predicting treatment outcome in psychological treatment services by identifying latent profiles of patients","volume":"197","author":"Saunders","year":"2016","journal-title":"J Affect Disord"},{"key":"ref47","doi-asserted-by":"publisher","first-page":"3231","DOI":"10.1017\/S0033291720005395","article-title":"Determinants of patient-reported outcome trajectories and symptomatic recovery in improving access to psychological therapies (IAPT) services","volume":"52","author":"Stochl","year":"2022","journal-title":"Psychol Med"},{"key":"ref48","doi-asserted-by":"publisher","first-page":"1347","DOI":"10.1056\/NEJMra1814259","article-title":"Machine learning in medicine","volume":"380","author":"Rajkomar","year":"2019","journal-title":"N Engl J Med"},{"key":"ref49","volume-title":"Machine learning for absolute beginners: A plain English introduction","author":"Theobald","year":"2017"},{"key":"ref50","author":"Coppersmith","year":"2016"},{"key":"ref51","doi-asserted-by":"crossref","DOI":"10.1145\/2858036.2858207","volume-title":"Discovering shifts to suicidal ideation from mental health content in social media","author":"De Choudhury","year":"2016"},{"key":"ref52","doi-asserted-by":"publisher","first-page":"68","DOI":"10.1016\/j.jpsychires.2014.08.017","article-title":"Quantitative forecasting of PTSD from early trauma responses: a machine learning application","volume":"59","author":"Galatzer-Levy","year":"2014","journal-title":"J Psychiatr Res"},{"key":"ref53","doi-asserted-by":"publisher","first-page":"1392","DOI":"10.1017\/S0033291720000227","article-title":"Machine learning enhances prediction of illness course: a longitudinal study in eating disorders","volume":"51","author":"Haynos","year":"2021","journal-title":"Psychol Med"},{"key":"ref54","volume-title":"Multi-task learning for predicting health, stress, and happiness","author":"Jaques","year":"2016"},{"key":"ref55","doi-asserted-by":"publisher","first-page":"1426","DOI":"10.1017\/S0033291719000151","article-title":"Machine learning in mental health: a scoping review of methods and applications","volume":"49","author":"Shatte","year":"2019","journal-title":"Psychol Med"},{"key":"ref56","first-page":"11","article-title":"Does machine learning really work?","volume":"18","author":"Mitchell","year":"1997","journal-title":"AI Mag"},{"key":"ref57","doi-asserted-by":"publisher","first-page":"94","DOI":"10.1016\/j.jpsychires.2016.03.016","article-title":"Combining clinical variables to optimize prediction of antidepressant treatment outcomes","volume":"78","author":"Iniesta","year":"2016","journal-title":"J Psychiatr Res"},{"key":"ref58","doi-asserted-by":"publisher","first-page":"255","DOI":"10.1126\/science.aaa8415","article-title":"Machine learning: trends, perspectives, and prospects","volume":"349","author":"Jordan","year":"2015","journal-title":"Science"},{"key":"ref59","doi-asserted-by":"publisher","first-page":"458","DOI":"10.1037\/met0000111","article-title":"A practical guide to big data research in psychology","volume":"21","author":"Chen","year":"2016","journal-title":"Psychol Methods"},{"key":"ref60","doi-asserted-by":"publisher","first-page":"92","DOI":"10.1080\/10503307.2020.1808729","article-title":"A scoping review of machine learning in psychotherapy research","volume":"31","author":"Aafjes-van Doorn","year":"2021","journal-title":"Psychother Res"},{"key":"ref61","doi-asserted-by":"publisher","first-page":"74","DOI":"10.3414\/ME00-01-0052","article-title":"Probability machines","volume":"51","author":"Malley","year":"2012","journal-title":"Methods Inf Med"},{"key":"ref62","doi-asserted-by":"publisher","first-page":"154","DOI":"10.1176\/appi.ajp.2016.16010077","article-title":"Predicting suicidal behavior from longitudinal electronic health records","volume":"174","author":"Barak-Corren","year":"2017","journal-title":"Am J Psychiatr"},{"key":"ref63","doi-asserted-by":"publisher","first-page":"457","DOI":"10.1177\/2167702617691560","article-title":"Predicting risk of suicide attempts over time through machine learning","volume":"5","author":"Walsh","year":"2017","journal-title":"Clin Psychol Sci"},{"key":"ref64","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1146\/annurev-clinpsy-032816-045037","article-title":"Machine learning approaches for clinical psychology and psychiatry","volume":"14","author":"Dwyer","year":"2018","journal-title":"Annu Rev Clin Psychol"},{"key":"ref65","doi-asserted-by":"publisher","first-page":"1100","DOI":"10.1177\/1745691617693393","article-title":"Choosing prediction over explanation in psychology: lessons from machine learning","volume":"12","author":"Yarkoni","year":"2017","journal-title":"Perspect Psychol Sci"},{"key":"ref66","volume-title":"Disease-atlas: Navigating disease trajectories using deep learning","author":"Lim","year":"2018"},{"key":"ref67","doi-asserted-by":"publisher","first-page":"116","DOI":"10.1038\/s41586-019-1390-1","article-title":"A clinically applicable approach to continuous prediction of future acute kidney injury","volume":"572","author":"Toma\u0161ev","year":"2019","journal-title":"Nature"},{"key":"ref68","doi-asserted-by":"publisher","first-page":"11242","DOI":"10.1038\/s41598-018-29523-2","article-title":"Prognostication and risk factors for cystic fibrosis via automated machine learning","volume":"8","author":"Alaa","year":"2018","journal-title":"Sci Rep"},{"key":"ref69","doi-asserted-by":"publisher","first-page":"855","DOI":"10.1002\/cpt.1482","article-title":"Pharmacogenomics-driven prediction of antidepressant treatment outcomes: a machine-learning approach with multi-trial replication","volume":"106","author":"Athreya","year":"2019","journal-title":"Clin Pharmacol Ther"},{"key":"ref70","doi-asserted-by":"publisher","first-page":"243","DOI":"10.1016\/S2215-0366(15)00471-X","article-title":"Cross-trial prediction of treatment outcome in depression: a machine learning approach","volume":"3","author":"Chekroud","year":"2016","journal-title":"Lancet Psychiatry"},{"key":"ref71","author":"Chernozhukov","year":"2018"},{"key":"ref72","doi-asserted-by":"publisher","first-page":"1433","DOI":"10.1002\/psp4.12715","article-title":"Heterogeneous treatment effect analysis based on machine-learning methodology","volume":"10","author":"Gong","year":"2021","journal-title":"CPT Pharmacometrics Syst Pharmacol"},{"key":"ref73","doi-asserted-by":"publisher","first-page":"103530","DOI":"10.1016\/j.brat.2019.103530","article-title":"Predicting cognitive behavioral therapy outcome in the outpatient sector based on clinical routine data: a machine learning approach","volume":"124","author":"Hilbert","year":"2020","journal-title":"Behav Res Ther"},{"key":"ref74","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1016\/j.jad.2020.03.081","article-title":"Predicting future suicidal behaviour in young adults, with different machine learning techniques: a population-based longitudinal study","volume":"271","author":"Van Mens","year":"2020","journal-title":"J Affect Disord"},{"key":"ref75","doi-asserted-by":"publisher","first-page":"e006556","DOI":"10.1161\/CIRCOUTCOMES.120.006556","article-title":"Recommendations for reporting machine learning analyses in clinical research","volume":"13","author":"Stevens","year":"2020","journal-title":"Circ Cardiovasc Qual Outcomes"},{"key":"ref76","doi-asserted-by":"publisher","first-page":"288","DOI":"10.1002\/cpp.529","article-title":"The outcome questionnaire (OQ-45) in a Dutch population: a cross-cultural validation","volume":"14","author":"De Jong","year":"2007","journal-title":"Clin Psychol Psychother"},{"key":"ref77","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1002\/cpp.572","article-title":"Evaluation of the mental health continuum\u2013short form (MHC\u2013SF) in setswana-speaking south Africans","volume":"15","author":"Keyes","year":"2008","journal-title":"Clin Psychol Psychother"},{"key":"ref78","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1002\/jclp.20741","article-title":"Evaluating the psychometric properties of the mental health continuum-short form (MHC-SF)","volume":"67","author":"Lamers","year":"2011","journal-title":"J Clin Psychol"},{"key":"ref79","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1007\/BF03062326","article-title":"De Outcome Questionnaire: psychometrische kenmerken van de Nederlandse vertaling","volume":"59","author":"De Jong","year":"2004","journal-title":"Ned Tijdschr Psychol Grensgebieden"},{"key":"ref80","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1007\/BF03077465","article-title":"De Nederlandse versie van de Outcome Questionnaire (OQ-45): een crossculturele validatie","volume":"36","author":"De Jong","year":"2008","journal-title":"Psychol Gezond"},{"key":"ref81","doi-asserted-by":"publisher","first-page":"2187","DOI":"10.1002\/jclp.22659","article-title":"Validation of the mental health continuum-short form and the dual continua model of well-being and psychopathology in an adult mental health setting","volume":"74","author":"Franken","year":"2018","journal-title":"J Clin Psychol"},{"key":"ref82","doi-asserted-by":"publisher","first-page":"582","DOI":"10.1097\/01.MLR.0000062554.74615.4C","article-title":"Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation","volume":"41","author":"Norman","year":"2003","journal-title":"Med Care"},{"key":"ref83","first-page":"19","article-title":"Statistical power analysis for the behavioral sciences: Jacob Cohen","volume":"84","author":"Cohen","year":"1988","journal-title":"J Am Stat Assoc"},{"key":"ref84","volume-title":"Released 2013. IBM SPSS statistics for windows, Version 27.0","year":"2020"},{"key":"ref85","volume-title":"R: A language and environment for statistical computing","year":"2020"},{"key":"ref86","unstructured":"KuhnM\n          2020"},{"key":"ref87","doi-asserted-by":"publisher","first-page":"112","DOI":"10.1093\/bioinformatics\/btr597","article-title":"MissForest\u2013non-parametric missing value imputation for mixed-type data","volume":"28","author":"Stekhoven","year":"2012","journal-title":"Bioinformatics"},{"key":"ref88","volume-title":"Pattern recognition and machine learning","author":"Bishop","year":"2006"},{"key":"ref89","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1037\/bul0000084","article-title":"Risk factors for suicidal thoughts and behaviors: a meta-analysis of 50 years of research","volume":"143","author":"Franklin","year":"2017","journal-title":"Psychol Bull"},{"key":"ref90","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4614-6849-3","volume-title":"Applied predictive modeling","author":"Kuhn","year":"2013"},{"key":"ref91","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4614-7138-7","volume-title":"An introduction to statistical learning","author":"James","year":"2013"},{"key":"ref92","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1002\/cpp.1868","article-title":"Patterns of change in different phases of outpatient psychotherapy: a stage-sequential pattern analysis of change in session reports","volume":"22","author":"Rubel","year":"2015","journal-title":"Clin Psychol Psychother"},{"key":"ref93","doi-asserted-by":"publisher","first-page":"e100614","DOI":"10.1371\/journal.pone.0100614","article-title":"Using early change to predict outcome in cognitive behaviour therapy: exploring timeframe, calculation method, and differences of disorder-specific versus general measures","volume":"9","author":"Schibbye","year":"2014","journal-title":"PLoS One"},{"key":"ref94","doi-asserted-by":"publisher","first-page":"289","DOI":"10.1093\/clipsy.6.3.289","article-title":"Rapid response to cognitive behavior therapy","volume":"6","author":"Wilson","year":"1999","journal-title":"Clin Gastroenterol Hepatol"},{"key":"ref95","first-page":"1","author":"Fonti","year":"2017"},{"key":"ref96","volume-title":"LASSO: a feature selection technique in predictive modeling for machine learning","author":"Muthukrishnan","year":"2016"},{"key":"ref97","doi-asserted-by":"crossref","DOI":"10.1002\/9781118548387","volume-title":"Applied logistic regression","author":"Hosmer","year":"2013"},{"key":"ref98","doi-asserted-by":"publisher","first-page":"1250","DOI":"10.1002\/eat.23510","article-title":"Prediction of eating disorder treatment response trajectories via machine learning does not improve performance versus a simpler regression approach","volume":"54","author":"Espel-Huynh","year":"2021","journal-title":"Int J Eat Disord"},{"key":"ref99","doi-asserted-by":"publisher","first-page":"e1576","DOI":"10.1002\/mpr.1576","article-title":"Prediction of outcome in internet-delivered cognitive behaviour therapy for paediatric obsessive-compulsive disorder: a machine learning approach","volume":"27","author":"Lenhard","year":"2018","journal-title":"Int J Methods Psychiatr Res"},{"key":"ref100","doi-asserted-by":"publisher","first-page":"101914","DOI":"10.1016\/j.nicl.2019.101914","article-title":"Application of machine learning to structural connectome to predict symptom reduction in depressed adolescents with cognitive behavioral therapy (CBT)","volume":"23","author":"Tymofiyeva","year":"2019","journal-title":"NeuroImage"},{"key":"ref101","doi-asserted-by":"publisher","first-page":"1254","DOI":"10.1038\/npp.2013.328","article-title":"Single-subject anxiety treatment outcome prediction using functional neuroimaging","volume":"39","author":"Ball","year":"2014","journal-title":"Neuropsychopharmacology"},{"key":"ref102","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1037\/1040-3590.18.2.133","article-title":"Empirically and clinically useful decision making in psychotherapy: differential predictions with treatment response models","volume":"18","author":"Lutz","year":"2006","journal-title":"Psychol Assess"},{"key":"ref103","doi-asserted-by":"publisher","first-page":"1818","DOI":"10.1001\/jama.2016.16435","article-title":"Confounding by indication in clinical research","volume":"316","author":"Kyriacou","year":"2016","journal-title":"JAMA"},{"key":"ref104","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511973000","volume-title":"Machine learning: The art and science of algorithms that make sense of data","author":"Flach","year":"2012"},{"key":"ref105","doi-asserted-by":"publisher","first-page":"257","DOI":"10.1002\/mpr.1423","article-title":"Reporting suicide attempts: consistency and its determinants in a large mental health study","volume":"23","author":"Eikelenboom","year":"2014","journal-title":"Int J Methods Psychiatr Res"},{"key":"ref106","doi-asserted-by":"publisher","first-page":"e0194889","DOI":"10.1371\/journal.pone.0194889","article-title":"Statistical and machine learning forecasting methods: concerns and ways forward","volume":"13","author":"Makridakis","year":"2018","journal-title":"PLoS One"},{"key":"ref107","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1109\/TSMCC.2006.876059","article-title":"Avoiding pitfalls in neural network research","volume":"37","author":"Zhang","year":"2006","journal-title":"IEEE Trans Syst Man Cybern Part C Appl Rev"},{"key":"ref108","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1016\/j.jclinepi.2019.02.004","article-title":"A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models","volume":"110","author":"Christodoulou","year":"2019","journal-title":"J Clin Epidemiol"},{"key":"ref109","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1080\/10503307.2020.1769219","article-title":"Personalized treatment selection in routine care: integrating machine learning and statistical algorithms to recommend cognitive behavioral or psychodynamic therapy","volume":"31","author":"Schwartz","year":"2021","journal-title":"Psychother Res"},{"key":"ref110","volume-title":"Pairing machine learning and clinical psychology: How you evaluate predictive performance matters","author":"Jacobucci","year":"2020"},{"key":"ref111","doi-asserted-by":"publisher","first-page":"214","DOI":"10.1002\/bsl.2392","article-title":"Machine learning in suicide science: applications and ethics","volume":"37","author":"Linthicum","year":"2019","journal-title":"Behav Sci Law"},{"key":"ref112","first-page":"1","volume-title":"Essential research findings in counselling and psychotherapy","author":"Cooper","year":"2008"},{"key":"ref113","doi-asserted-by":"publisher","first-page":"1157","DOI":"10.1002\/jclp.10044","article-title":"Do early responders to psychotherapy maintain treatment gains?","volume":"58","author":"Haas","year":"2002","journal-title":"J Clin Psychol"},{"key":"ref114","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1186\/1471-2458-13-119","article-title":"Positive psychology interventions: a meta-analysis of randomized controlled studies","volume":"13","author":"Bolier","year":"2013","journal-title":"BMC Public Health"},{"key":"ref115","doi-asserted-by":"publisher","first-page":"442","DOI":"10.1037\/cps0000074","article-title":"Assessing mental wellbeing using the mental health continuum\u2014short form: a systematic review and meta-analytic structural equation modelling","volume":"29","author":"Iasiello","year":"2022","journal-title":"Clin Psychol Sci Pract"},{"key":"ref116","doi-asserted-by":"publisher","first-page":"152189","DOI":"10.1016\/j.comppsych.2020.152189","article-title":"The relationship between emotion regulation and well-being in patients with mental disorders: a meta-analysis","volume":"102","author":"Kraiss","year":"2020","journal-title":"Compr Psychiatry"},{"key":"ref117","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/1472-6963-10-26","article-title":"Mental illness and well-being: the central importance of positive psychology and recovery approaches","volume":"10","author":"Slade","year":"2010","journal-title":"BMC Health Serv Res"},{"key":"ref118","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1016\/j.brat.2017.01.008","article-title":"Both positive mental health and psychopathology should be monitored in psychotherapy: confirmation for the dual-factor model in acceptance and commitment therapy","volume":"91","author":"Trompetter","year":"2017","journal-title":"Behav Res Ther"},{"key":"ref119","doi-asserted-by":"publisher","first-page":"946","DOI":"10.1002\/eat.22411","article-title":"Predictors of treatment outcome in individuals with eating disorders: a systematic review and meta-analysis","volume":"48","author":"Vall","year":"2015","journal-title":"Int J Eat Disord"},{"key":"ref120","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1002\/jclp.20764","article-title":"What works for whom: tailoring psychotherapy to the person","volume":"67","author":"Norcross","year":"2011","journal-title":"J Clin Psychol"},{"key":"ref121","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1002\/cpp.713","article-title":"Patterns of inner change and their relation with patient characteristics and outcome in a psychoanalytic hospitalization-based treatment for personality disordered patients","volume":"18","author":"Vermote","year":"2011","journal-title":"Clin Psychol Psychother"},{"key":"ref122","volume-title":"Encyclopedia of machine learning","author":"Sammut","year":"2011"},{"key":"ref123","volume-title":"Alexithymia in borderline personality pathology: From theory to a biosensor application","author":"Derks","year":"2022"},{"key":"ref124","first-page":"413","article-title":"Smart sensing enhanced diagnostic expert systems","volume-title":"Digital phenotyping and Mobile sensing: New developments in Psychoinformatics","author":"Terhorst","year":"2022"},{"key":"ref125","doi-asserted-by":"publisher","first-page":"23","DOI":"10.1146\/annurev-clinpsy-032816-044949","article-title":"Personal sensing: understanding mental health using ubiquitous sensors and machine learning","volume":"13","author":"Mohr","year":"2017","journal-title":"Annu Rev Clin Psychol"},{"key":"ref126","doi-asserted-by":"publisher","first-page":"625247","DOI":"10.3389\/fpsyt.2021.625247","article-title":"Predicting symptoms of depression and anxiety using smartphone and wearable data","volume":"12","author":"Moshe","year":"2021","journal-title":"Front Psych"},{"key":"ref127","doi-asserted-by":"publisher","first-page":"1440","DOI":"10.1002\/acr.24052","article-title":"Longitudinal study of fatigue, stress, and depression: role of reduction in stress toward improvement in fatigue","volume":"72","author":"Azizoddin","year":"2020","journal-title":"Arthritis Care Res"},{"key":"ref128","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1016\/j.genhosppsych.2016.07.004","article-title":"Distribution-based estimates of minimal important difference for hospital anxiety and depression scale and impact of event scale-revised in survivors of acute respiratory failure","volume":"42","author":"Chan","year":"2016","journal-title":"Gen Hosp Psychiatry"},{"key":"ref129","doi-asserted-by":"publisher","first-page":"102873","DOI":"10.1016\/j.ajp.2021.102873","article-title":"Sensitivity to change and minimal clinically important difference of Edinburgh postnatal depression scale","volume":"66","author":"Mao","year":"2021","journal-title":"Asian J Psychiatr"},{"key":"ref130","doi-asserted-by":"publisher","first-page":"83","DOI":"10.1002\/da.20505","article-title":"Nonresponse, partial response, and failure to achieve remission: humanistic and cost burden in major depressive disorder","volume":"26","author":"Mauskopf","year":"2009","journal-title":"Depress Anxiety"},{"key":"ref131","doi-asserted-by":"publisher","first-page":"174","DOI":"10.1016\/j.jpsychores.2014.06.015","article-title":"Relationships of diabetes-specific emotional distress, depression, anxiety, and overall well-being with HbA1c in adult persons with type 1 diabetes","volume":"77","author":"Strandberg","year":"2014","journal-title":"J Psychosom Res"},{"key":"ref132","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1037\/0022-006X.59.1.12","article-title":"Clinical significance: a statistical approach to defining meaningful change in psychotherapy research","volume":"59","author":"Jacobson","year":"1991","journal-title":"J Cons Clin Psychol"},{"key":"ref133","doi-asserted-by":"crossref","DOI":"10.1093\/med-psych\/9780190843953.003.0002","article-title":"Alliance in adult psychotherapy","volume-title":"Psychotherapy relationships that work: Evidence-based therapist contributions","author":"Fl\u00fcckiger","year":"2019"},{"key":"ref134","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1002\/cpt.1907","article-title":"From real-world patient data to individualized treatment effects using machine learning: current and future methods to address underlying challenges","volume":"109","author":"Bica","year":"2021","journal-title":"Clin Pharmacol Ther"},{"key":"ref135","doi-asserted-by":"publisher","first-page":"231","DOI":"10.1136\/bmjqs-2018-008370","article-title":"Artificial intelligence, bias and clinical safety","volume":"28","author":"Challen","year":"2019","journal-title":"BMJ Qual Saf"},{"key":"ref136","doi-asserted-by":"publisher","first-page":"e2032539","DOI":"10.1001\/jamanetworkopen.2020.32539","article-title":"Building predictive models for clinical care\u2014where to build and what to predict?","volume":"4","author":"Selby","year":"2021","journal-title":"JAMA Netw Open"},{"key":"ref137","doi-asserted-by":"publisher","first-page":"947081","DOI":"10.3389\/fpsyt.2023.947081","article-title":"Prediction of Chinese clients\u2019 satisfaction with psychotherapy by machine learning","volume":"14","author":"Yao","year":"2023","journal-title":"Front Psych"},{"key":"ref138","doi-asserted-by":"publisher","first-page":"323","DOI":"10.1037\/a0016973","article-title":"An introduction to recursive partitioning: rationale, application, and characteristics of classification and regression trees, bagging, and random forests","volume":"14","author":"Strobl","year":"2009","journal-title":"Psychol Methods"},{"key":"ref139","doi-asserted-by":"publisher","first-page":"1265","DOI":"10.4088\/JCP.12l07879","article-title":"Patient adherence to cognitive-behavioral therapy predicts long-term outcome in obsessive-compulsive disorder","volume":"73","author":"Simpson","year":"2012","journal-title":"J Clin Psychiatry"},{"key":"ref140","volume-title":"What works when for whom?: A methodological reflection on therapeutic change process research","author":"Smink","year":"2021"},{"key":"ref141","doi-asserted-by":"publisher","first-page":"575931","DOI":"10.3389\/fpsyt.2021.575931","article-title":"Analysis of the emails from the Dutch web-based intervention \u201calcohol de baas\u201d: assessment of early indications of drop-out in an online alcohol abuse intervention","volume":"12","author":"Smink","year":"2021","journal-title":"Front Psych"},{"key":"ref142","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12888-019-2214-4","article-title":"Lower versus higher frequency of sessions in starting outpatient mental health care and the risk of a chronic course; a naturalistic cohort study","volume":"19","author":"Tiemens","year":"2019","journal-title":"BMC Psychiatry"},{"key":"ref143","doi-asserted-by":"publisher","first-page":"102002","DOI":"10.1016\/j.cpr.2021.102002","article-title":"Using progress feedback to improve outcomes and reduce drop-out, treatment duration, and deterioration: a multilevel meta-analysis","volume":"85","author":"De Jong","year":"2021","journal-title":"Clin Psychol Rev"},{"key":"ref144","doi-asserted-by":"publisher","first-page":"269","DOI":"10.1007\/s10488-022-01236-9","article-title":"A systematic review and meta-analysis of measurement feedback systems in treatment for common mental health disorders","volume":"50","author":"Rognstad","year":"2023","journal-title":"Adm Policy Ment Health Ment Health Serv Res"},{"key":"ref145","doi-asserted-by":"publisher","first-page":"137","DOI":"10.1080\/10503307.2018.1563312","article-title":"A demonstration of a multi-method variable selection approach for treatment selection: recommending cognitive\u2013behavioral versus psychodynamic therapy for mild to moderate adult depression","volume":"30","author":"Cohen","year":"2020","journal-title":"Psychother Res"},{"key":"ref146","doi-asserted-by":"publisher","first-page":"262","DOI":"10.30773\/pi.2018.12.21.2","article-title":"Review of machine learning algorithms for diagnosing mental illness","volume":"16","author":"Cho","year":"2019","journal-title":"Psychiatry Investig"},{"key":"ref147","doi-asserted-by":"publisher","first-page":"843","DOI":"10.1002\/cpt.1658","article-title":"Examining the use of real-world evidence in the regulatory process","volume":"107","author":"Beaulieu-Jones","year":"2020","journal-title":"Clin Pharmacol Ther"}],"container-title":["Frontiers in Psychiatry"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpsyt.2023.1236551\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,26]],"date-time":"2023-09-26T00:45:10Z","timestamp":1695689110000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpsyt.2023.1236551\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,25]]},"references-count":147,"alternative-id":["10.3389\/fpsyt.2023.1236551"],"URL":"https:\/\/doi.org\/10.3389\/fpsyt.2023.1236551","relation":{},"ISSN":["1664-0640"],"issn-type":[{"value":"1664-0640","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,25]]},"article-number":"1236551"}}