{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,19]],"date-time":"2026-08-19T18:09:09Z","timestamp":1787162949548,"version":"build-2736575974"},"reference-count":45,"publisher":"Oxford University Press (OUP)","issue":"6","funder":[{"DOI":"10.13039\/100000025","name":"National Institute of Mental Health","doi-asserted-by":"publisher","award":["R01MH135861"],"award-info":[{"award-number":["R01MH135861"]}],"id":[{"id":"10.13039\/100000025","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000025","name":"NIMH","doi-asserted-by":"publisher","award":["R21\/R33MH096828"],"award-info":[{"award-number":["R21\/R33MH096828"]}],"id":[{"id":"10.13039\/100000025","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000025","name":"NIMH","doi-asserted-by":"publisher","award":["R01MH79217"],"award-info":[{"award-number":["R01MH79217"]}],"id":[{"id":"10.13039\/100000025","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000025","name":"NIMH","doi-asserted-by":"publisher","award":["R01MH079154"],"award-info":[{"award-number":["R01MH079154"]}],"id":[{"id":"10.13039\/100000025","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000025","name":"NIMH","doi-asserted-by":"publisher","award":["R01MH079377"],"award-info":[{"award-number":["R01MH079377"]}],"id":[{"id":"10.13039\/100000025","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000025","name":"NIMH","doi-asserted-by":"publisher","award":["R01MH064188"],"award-info":[{"award-number":["R01MH064188"]}],"id":[{"id":"10.13039\/100000025","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000025","name":"NIMH","doi-asserted-by":"publisher","award":["R01MH055126"],"award-info":[{"award-number":["R01MH055126"]}],"id":[{"id":"10.13039\/100000025","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000025","name":"NIMH","doi-asserted-by":"publisher","award":["R01MH055121"],"award-info":[{"award-number":["R01MH055121"]}],"id":[{"id":"10.13039\/100000025","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000025","name":"NIMH","doi-asserted-by":"publisher","award":["R01MH055121"],"award-info":[{"award-number":["R01MH055121"]}],"id":[{"id":"10.13039\/100000025","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000025","name":"NIMH","doi-asserted-by":"publisher","award":["R01MH055126"],"award-info":[{"award-number":["R01MH055126"]}],"id":[{"id":"10.13039\/100000025","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Intramural Research Program of the National Institute of Mental Health","award":["ZIC-MH002968"],"award-info":[{"award-number":["ZIC-MH002968"]}]},{"name":"Intramural Research Program of the National Institute of Mental Health","award":["ZIA-MH002781"],"award-info":[{"award-number":["ZIA-MH002781"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,11,3]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>To develop and evaluate an automated classification system for labeling Exposure Process Coding System (EPCS) quality codes\u2014specifically exposure and encourage events\u2014during in-person exposure therapy sessions using automatic speech recognition (ASR) and natural language processing techniques.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and Methods<\/jats:title>\n                    <jats:p>The system was trained and tested on 360 manually labeled pediatric Obsessive-Compulsive Disorder (OCD) therapy sessions from 3 clinical trials. Audio recordings were transcribed using ASR tools (OpenAI\u2019s Whisper and Google Speech-to-Text). Transcription accuracy was evaluated via word error rate (WER) on manual transcriptions of 2-minute audio segments compared against ASR-generated transcripts. The resulting text was analyzed with transformer-based models, including Bidirectional Encoder Representations from Transformers (BERT), Sentence-BERT, and Meta Llama 3. Models were trained to predict EPCS codes in 2 classification settings: sequence-level classification, where events are labeled in delimited text chunks, and token-level classification, where event boundaries are unknown. Classification was performed either with fine-tuned transformer-based models, or with logistic regression on embeddings produced by each model.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>With respect to transcription accuracy, Whisper outperformed Google Speech-to-Text with a lower WER (0.31 vs 0.51). For sequence classification setting, Llama 3 models achieved high performance with area under the ROC curve (AUC) scores of 0.95 for exposures and 0.75 for encourage events, outperforming traditional methods and standard BERT models. In the token-level setting, fine-tuned BERT models performed best, achieving AUC scores of 0.85 for exposures and 0.75 for encourage events.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion and Conclusion<\/jats:title>\n                    <jats:p>Current ASR and transformer-based models enable automated quality coding of in-person exposure therapy sessions. These findings demonstrate potential for real-time assessment in clinical practice and scalable research on effective therapy methods. Future work should focus on optimization, including improvements in ASR accuracy, expanding training datasets, and multimodal data integration.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamiaopen\/ooaf127","type":"journal-article","created":{"date-parts":[[2025,10,6]],"date-time":"2025-10-06T11:30:20Z","timestamp":1759750220000},"source":"Crossref","is-referenced-by-count":4,"title":["Automated classification of exposure and encourage events in speech data from pediatric OCD treatment"],"prefix":"10.1093","volume":"8","author":[{"given":"Juan Antonio","family":"Lossio-Ventura","sequence":"first","affiliation":[{"name":"Machine Learning Core, National Institute of Mental Health, National Institutes of Health , Bethesda, MD 20892,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Samuel","family":"Frank","sequence":"additional","affiliation":[{"name":"Emotion and Development Branch, National Institute of Mental Health, National Institutes of Health , Bethesda, MD 20892,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Grace","family":"Ringlein","sequence":"additional","affiliation":[{"name":"Emotion and Development Branch, National Institute of Mental Health, National Institutes of Health , Bethesda, MD 20892,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kirsten","family":"Bonson","sequence":"additional","affiliation":[{"name":"Department of Electrical and Biomedical Engineering, University of Vermont , Burlington, VT 05405,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ardyn","family":"Olszko","sequence":"additional","affiliation":[{"name":"Department of Electrical and Biomedical Engineering, University of Vermont , Burlington, VT 05405,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Abbey","family":"Knobel","sequence":"additional","affiliation":[{"name":"Department of Electrical and Biomedical Engineering, University of Vermont , Burlington, VT 05405,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel S","family":"Pine","sequence":"additional","affiliation":[{"name":"Emotion and Development Branch, National Institute of Mental Health, National Institutes of Health , Bethesda, MD 20892,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jennifer B","family":"Freeman","sequence":"additional","affiliation":[{"name":"Warren Alpert Medical School, Brown University Psychiatry and Human Behavior, , East Providence, RI 02915,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kristen","family":"Benito","sequence":"additional","affiliation":[{"name":"Warren Alpert Medical School, Brown University Psychiatry and Human Behavior, , East Providence, RI 02915,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David C","family":"Jangraw","sequence":"additional","affiliation":[{"name":"Emotion and Development Branch, National Institute of Mental Health, National Institutes of Health , Bethesda, MD 20892,","place":["United States"]},{"name":"Department of Electrical and Biomedical Engineering, University of Vermont , Burlington, VT 05405,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Francisco","family":"Pereira","sequence":"additional","affiliation":[{"name":"Machine Learning Core, National Institute of Mental Health, National Institutes of Health , Bethesda, MD 20892,","place":["United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2025,12,9]]},"reference":[{"key":"2025121107225504300_ooaf127-B1","doi-asserted-by":"crossref","first-page":"557","DOI":"10.1016\/j.psc.2010.04.002","article-title":"Efficacy of cognitive behavioral therapy for anxiety disorders: a review of meta-analytic findings","volume":"33","author":"Olatunji","year":"2010","journal-title":"Psychiatr Clin North Am"},{"key":"2025121107225504300_ooaf127-B2","doi-asserted-by":"crossref","first-page":"502","DOI":"10.1002\/da.22728","article-title":"Cognitive behavioral therapy for anxiety and related disorders: a meta-analysis of randomized placebo-controlled trials","volume":"35","author":"Carpenter","year":"2018","journal-title":"Depress Anxiety"},{"key":"2025121107225504300_ooaf127-B3","doi-asserted-by":"crossref","first-page":"1565","DOI":"10.1017\/S0033291718002234","article-title":"Efficacy of exposure-based cognitive behaviour therapy for post-traumatic stress disorder in emergency service personnel: a randomised clinical trial","volume":"49","author":"Bryant","year":"2019","journal-title":"Psychol Med"},{"key":"2025121107225504300_ooaf127-B4","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1016\/j.janxdis.2016.04.001","article-title":"Factors associated with practitioners\u2019 use of exposure therapy for childhood anxiety disorders","volume":"40","author":"Whiteside","year":"2016","journal-title":"J Anxiety Disord"},{"key":"2025121107225504300_ooaf127-B5","doi-asserted-by":"crossref","first-page":"1578","DOI":"10.1037\/0003-066X.47.12.1578","article-title":"The lab versus the clinic","volume":"47","author":"Weisz","year":"1992","journal-title":"Am Psychol"},{"key":"2025121107225504300_ooaf127-B6","volume-title":"Handbook of Evidence-Based Therapies for Children and Adolescents: Bridging Science and Practice","author":"Higa","year":"2008"},{"key":"2025121107225504300_ooaf127-B7","doi-asserted-by":"crossref","first-page":"829","DOI":"10.1037\/0022-006X.75.6.829","article-title":"Treatment integrity in psychotherapy research: analysis of the studies and examination of the associated factors","volume":"75","author":"Perepletchikova","year":"2007","journal-title":"J Consult Clin Psychol"},{"key":"2025121107225504300_ooaf127-B8","doi-asserted-by":"crossref","first-page":"1000","DOI":"10.1001\/jama.280.11.1000","article-title":"The urgent need to improve health care quality. Institute of medicine national roundtable on health care quality","volume":"280","author":"Chassin","year":"1998","journal-title":"J Am Med Assoc"},{"key":"2025121107225504300_ooaf127-B9","year":"2010"},{"key":"2025121107225504300_ooaf127-B10","author":"Centers for Medicare & Medicaid Services","year":"2024"},{"key":"2025121107225504300_ooaf127-B11","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1016\/j.jocrd.2012.01.001","article-title":"CBT specific process in exposure-based treatments: initial examination in a pediatric OCD sample","volume":"1","author":"Benito","year":"2012","journal-title":"J Obsessive Compuls Relat Disord"},{"key":"2025121107225504300_ooaf127-B12","doi-asserted-by":"crossref","first-page":"615","DOI":"10.1037\/ccp0000315","article-title":"Measuring fear change within exposures: functionally-defined habituation predicts outcome in three randomized controlled trials for pediatric OCD","volume":"86","author":"Benito","year":"2018","journal-title":"J Consult Clin Psychol"},{"key":"2025121107225504300_ooaf127-B13","doi-asserted-by":"crossref","first-page":"523","DOI":"10.1016\/j.beth.2020.07.004","article-title":"Therapist behavior during exposure tasks predicts habituation and clinical outcome in three randomized controlled trials for pediatric OCD","volume":"52","author":"Benito","year":"2021","journal-title":"Behav Ther"},{"key":"2025121107225504300_ooaf127-B14","first-page":"4171","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Devlin","year":"2019"},{"key":"2025121107225504300_ooaf127-B15","first-page":"1218","volume-title":"Proceedings of the 20th Chinese National Conference on Computational Linguistics","author":"Zhuang","year":"2021"},{"key":"2025121107225504300_ooaf127-B16","author":"Sanh","year":"2019"},{"key":"2025121107225504300_ooaf127-B17","author":"GPT","year":"2024"},{"key":"2025121107225504300_ooaf127-B18","author":"Grattafiori","year":"2024"},{"key":"2025121107225504300_ooaf127-B19","author":"Meta","year":"2024"},{"key":"2025121107225504300_ooaf127-B20","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1038\/s41398-023-02592-2","article-title":"Natural language processing for mental health interventions: a systematic review and research framework","volume":"13","author":"Malgaroli","year":"2023","journal-title":"Transl Psychiatry"},{"key":"2025121107225504300_ooaf127-B21","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1037\/pst0000517","article-title":"Leveraging natural language processing to study emotional coherence in psychotherapy","volume":"61","author":"Atzil-Slonim","year":"2024","journal-title":"Psychotherapy"},{"key":"2025121107225504300_ooaf127-B22","doi-asserted-by":"crossref","first-page":"ocae298","DOI":"10.1093\/jamia\/ocae298","article-title":"Using large language models to detect outcomes in qualitative studies of adolescent depression","author":"Xin","year":"2024","journal-title":"J Am Med Inform Assoc"},{"key":"2025121107225504300_ooaf127-B23","doi-asserted-by":"crossref","first-page":"166","DOI":"10.1038\/s41398-025-03379-3","article-title":"COMPASS: computational mapping of patient-therapist alliance strategies with language modeling","volume":"15","author":"Lin","year":"2025","journal-title":"Transl Psychiatry"},{"key":"2025121107225504300_ooaf127-B24","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1001\/jamapsychiatry.2019.2664","article-title":"Quantifying the association between psychotherapy content and clinical outcomes using deep learning","volume":"77","author":"Ewbank","year":"2020","journal-title":"JAMA Psychiatry"},{"key":"2025121107225504300_ooaf127-B25","doi-asserted-by":"crossref","first-page":"1043","DOI":"10.1016\/j.jaac.2017.10.001","article-title":"Therapist-reported features of exposure tasks that predict differential treatment outcomes for youth with anxiety","volume":"56","author":"Peris","year":"2017","journal-title":"J Am Acad Child Adolesc Psychiatry"},{"key":"2025121107225504300_ooaf127-B26","first-page":"1969","article-title":"Pediatric OCD treatment study (POTS) team. Cognitive-behavior therapy, sertraline, and their combination for children and adolescents with obsessive-compulsive disorder: the pediatric OCD treatment study (POTS) randomized controlled trial","volume":"292","author":"Team","year":"2004","journal-title":"Arch Psychol"},{"key":"2025121107225504300_ooaf127-B27","first-page":"1224","article-title":"Cognitive behavior therapy augmentation of pharmacotherapy in pediatric obsessive-compulsive disorder: the pediatric OCD treatment study II (POTS II) randomized controlled trial","volume":"306","author":"Franklin","year":"2011","journal-title":"Arch Psychol"},{"key":"2025121107225504300_ooaf127-B28","first-page":"689","article-title":"Family-based treatment of early childhood obsessive-compulsive disorder: the pediatric obsessive-compulsive disorder treatment study for young children (POTS Jr)\u2013a randomized clinical trial","volume":"71","author":"Freeman","year":"2014","journal-title":"Arch Psychol"},{"key":"2025121107225504300_ooaf127-B29","doi-asserted-by":"crossref","first-page":"276","DOI":"10.11613\/BM.2012.031","article-title":"Interrater reliability: the kappa statistic","volume":"22","author":"McHugh","year":"2012","journal-title":"Biochem Med (Zagreb)"},{"key":"2025121107225504300_ooaf127-B30","first-page":"28492","author":"Radford","year":"2023"},{"key":"2025121107225504300_ooaf127-B31","author":"Whisper. [Online]","year":"2023"},{"key":"2025121107225504300_ooaf127-B32","author":"Google Speech-to-Text","year":"2023"},{"key":"2025121107225504300_ooaf127-B33","first-page":"3982","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Reimers","year":"2019"},{"key":"2025121107225504300_ooaf127-B34","author":"Liu","year":"2019"},{"key":"2025121107225504300_ooaf127-B35","first-page":"2825","article-title":"Scikit-learn: machine learning in python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J Mach Learning Res"},{"key":"2025121107225504300_ooaf127-B36","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1016\/j.ipm.2013.08.006","article-title":"The impact of preprocessing on text classification","volume":"50","author":"Uysal","year":"2014","journal-title":"Inf Process Manage"},{"key":"2025121107225504300_ooaf127-B37","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1007\/s10588-018-9266-8","article-title":"The impact of preprocessing steps on the accuracy of machine learning algorithms in sentiment analysis","volume":"25","author":"Alam","year":"2019","journal-title":"Comput Math Org Theory"},{"key":"2025121107225504300_ooaf127-B38","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1017\/S1351324922000213","article-title":"Comparison of text preprocessing methods","volume":"29","author":"Chai","year":"2023","journal-title":"Nat Lang Eng"},{"key":"2025121107225504300_ooaf127-B39","author":"Natural Language Toolkit","year":"2024"},{"key":"2025121107225504300_ooaf127-B40","author":"spaCy","year":"2024"},{"key":"2025121107225504300_ooaf127-B41","first-page":"23","article-title":"A new algorithm for data compression","volume":"12","author":"Gage","year":"1994","journal-title":"C Users J"},{"key":"2025121107225504300_ooaf127-B42","doi-asserted-by":"crossref","first-page":"1455","DOI":"10.1121\/1.426686","article-title":"Acoustics of children\u2019s speech: developmental changes of temporal and spectral parameters","volume":"105","author":"Lee","year":"1999","journal-title":"J Acoust Soc Am"},{"key":"2025121107225504300_ooaf127-B43","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1016\/j.learninstruc.2016.04.003","article-title":"Off-task behavior in elementary school children","volume":"44","author":"Godwin","year":"2016","journal-title":"Learn Instr"},{"key":"2025121107225504300_ooaf127-B44","doi-asserted-by":"crossref","first-page":"1015","DOI":"10.1093\/jamia\/ocaf045","article-title":"Evaluating the effectiveness of biomedical fine-tuning for large language models on clinical tasks","volume":"32","author":"Dorfner","year":"2025","journal-title":"J Am Med Inf Assoc"},{"key":"2025121107225504300_ooaf127-B45","doi-asserted-by":"crossref","first-page":"687","DOI":"10.1177\/21677026241240727","article-title":"Mechanisms of change in exposure therapy for anxiety and related disorders: a research agenda","volume":"13","author":"Benito","year":"2025","journal-title":"Clin Psychol Sci"}],"container-title":["JAMIA Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article-pdf\/8\/6\/ooaf127\/65835691\/ooaf127.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article-pdf\/8\/6\/ooaf127\/65835691\/ooaf127.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,11]],"date-time":"2025-12-11T12:23:07Z","timestamp":1765455787000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article\/doi\/10.1093\/jamiaopen\/ooaf127\/8377101"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,3]]},"references-count":45,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2025,11,3]]}},"URL":"https:\/\/doi.org\/10.1093\/jamiaopen\/ooaf127","relation":{},"ISSN":["2574-2531"],"issn-type":[{"value":"2574-2531","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,12]]},"published":{"date-parts":[[2025,11,3]]},"article-number":"ooaf127"}}