{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,26]],"date-time":"2026-08-26T16:25:02Z","timestamp":1787761502302,"version":"build-2784847793"},"reference-count":74,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Ment Health"],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Internet-based cognitive behavioral therapy (iCBT) is an effective and scalable alternative to face-to-face psychotherapy, but its reach is constrained by the time therapists spend reviewing patient input and manually drafting written responses. Studies suggest that large language models (LLMs) may be capable of generating high-quality therapeutic text, with the potential to support therapists in delivering treatment. Their suitability as therapist-support tools in structured iCBT, however, remains insufficiently studied.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>This study aims to assess the quality of LLM-generated iCBT responses to patient messages by comparing them to the quality of responses produced by humans.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>In a preregistered blinded clinician rating experiment, experienced clinicians assessed the quality of human-produced vs LLM-generated therapist responses within a simulated iCBT treatment for functional somatic disorder. Raters were exposed to a stimulus material consisting of 5 fictitious patient messages, each paired with 1 human and 1 LLM-generated response. Raters assessed message\/response pairs on 5 quality dimensions (overall quality, helpfulness, empathy, professionalism, and protocol adherence) and were asked to indicate the source of the response (human\/LLM). Analyses were primarily descriptive, supplemented by exploratory statistical tests and descriptive thematic content analysis of open-ended text fields. The full preregistered study protocol is available at Open Science Framework.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>A total of 61 raters provided data, while 54 were eligible and included for analysis. Human- and LLM-generated responses were rated similarly across quality dimensions on a 1-5 scale: overall quality (LLM: mean 4.00, SD 0.54 vs human: mean 3.96, SD 0.53; d=0.06), helpfulness (LLM: mean 3.85, SD 0.57 vs human: mean 3.93, SD 0.49; d=0.13), professionalism (LLM: mean 4.25, SD 0.53 vs human: mean 4.11, SD 0.53; d=0.24), protocol adherence (LLM: mean 4.13, SD 0.52 vs human: mean 4.13, SD 0.54; d=0.03) and empathy (LLM: mean 4.31, SD 0.47 vs human: mean 4.08, SD 0.50; d=0.42). Raters correctly identified the source of human-generated responses (mean 79%, SD 19.65%) more accurately than LLM-generated responses (mean 63%, SD 21.30%). In all, 30\/54 (55%) raters responded to one or more open text fields. Qualitative analysis indicated that LLM-generated responses were perceived as polished but also generic and at times excessively empathetic.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>LLM-generated responses were judged to be of comparable quality to those written by human therapists, though qualitative feedback indicated they were at times generic and insufficiently challenging. These findings provide initial support for the feasibility of using LLMs as therapist-support tools in iCBT, but further research is needed to determine whether their integration yields tangible clinical and organizational benefits.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.2196\/96835","type":"journal-article","created":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T13:10:07Z","timestamp":1783429807000},"page":"e96835","source":"Crossref","is-referenced-by-count":0,"title":["Large Language Models as a New Tool for Therapists in Internet-Based Cognitive Behavioral Therapy: Blinded Clinician Rating Pilot Experiment"],"prefix":"10.2196","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-4335-5640","authenticated-orcid":false,"given":"Thomas Tandrup","family":"Lamm","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-0585-2599","authenticated-orcid":false,"given":"Arthur Bran","family":"Herbener","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3964-5514","authenticated-orcid":false,"given":"Oliver R\u00f8nn","family":"Christensen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9352-6703","authenticated-orcid":false,"given":"Malene Flensborg","family":"Damholdt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5882-5118","authenticated-orcid":false,"given":"Kaare Bro","family":"Wellnitz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8612-5466","authenticated-orcid":false,"given":"Heidi Fr\u00f8lund","family":"Pedersen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9683-7416","authenticated-orcid":false,"given":"Lisbeth","family":"Frostholm","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,8,26]]},"reference":[{"key":"ref1","unstructured":"World mental health today: latest dataWorld Health Organization202509022026-08-21https:\/\/www.who.int\/publications\/i\/item\/9789240113817"},{"key":"ref2","doi-asserted-by":"publisher","DOI":"10.1002\/wps.70002"},{"key":"ref3","doi-asserted-by":"publisher","DOI":"10.1016\/j.invent.2026.100927"},{"key":"ref4","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-clinpsy-021815-093006"},{"key":"ref5","doi-asserted-by":"publisher","DOI":"10.1002\/wps.21088"},{"key":"ref6","doi-asserted-by":"publisher","DOI":"10.1016\/j.invent.2018.07.006"},{"key":"ref7","doi-asserted-by":"publisher","DOI":"10.1177\/09637214251386047"},{"key":"ref8","doi-asserted-by":"publisher","DOI":"10.1038\/s44184-024-00056-z"},{"key":"ref9","doi-asserted-by":"publisher","DOI":"10.2196\/82369"},{"key":"ref10","doi-asserted-by":"publisher","DOI":"10.2196\/59479"},{"key":"ref11","doi-asserted-by":"publisher","DOI":"10.2196\/91746"},{"key":"ref12","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2023.1281255"},{"key":"ref13","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2024.2385001"},{"key":"ref14","doi-asserted-by":"publisher","DOI":"10.1016\/j.chbah.2024.100077"},{"key":"ref15","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-026-04278-w"},{"key":"ref16","doi-asserted-by":"publisher","DOI":"10.1016\/j.invent.2025.100841"},{"key":"ref17","doi-asserted-by":"publisher","DOI":"10.1038\/s44271-024-00182-6"},{"key":"ref18","doi-asserted-by":"publisher","DOI":"10.1097\/YCO.0000000000001048"},{"key":"ref19","doi-asserted-by":"publisher","DOI":"10.1002\/jcpy.1393"},{"key":"ref20","doi-asserted-by":"publisher","DOI":"10.2196\/80739"},{"key":"ref21","doi-asserted-by":"publisher","DOI":"10.1126\/science.aec8352"},{"key":"ref22","doi-asserted-by":"publisher","DOI":"10.1038\/s44220-026-00595-8"},{"key":"ref23","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01232-3"},{"key":"ref24","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2024.10.356"},{"key":"ref25","doi-asserted-by":"publisher","DOI":"10.1016\/j.invent.2025.100875"},{"key":"ref26","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2024.38573"},{"key":"ref27","doi-asserted-by":"publisher","DOI":"10.1038\/s44401-025-00032-5"},{"key":"ref28","doi-asserted-by":"publisher","DOI":"10.1017\/gmh.2024.94"},{"key":"ref29","doi-asserted-by":"publisher","DOI":"10.1186\/s44247-024-00105-9"},{"key":"ref30","doi-asserted-by":"publisher","DOI":"10.1145\/3710993"},{"key":"ref31","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-022-00593-2"},{"key":"ref32","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.2319112121"},{"key":"ref33","doi-asserted-by":"publisher","DOI":"10.1186\/s12916-020-1505-4"},{"key":"ref34","unstructured":"Large language models as a new tool for iCBT therapists? An experimental pilot studyOpen Science Framework2026-08-15https:\/\/osf.io\/yxncv"},{"key":"ref35","doi-asserted-by":"publisher","DOI":"10.1145\/3589334.3645643"},{"key":"ref36","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.560"},{"key":"ref37","doi-asserted-by":"publisher","DOI":"10.2196\/69857"},{"key":"ref38","doi-asserted-by":"publisher","DOI":"10.1016\/j.conctc.2023.101069"},{"key":"ref39","doi-asserted-by":"publisher","DOI":"10.1080\/16506073.2018.1457079"},{"key":"ref40","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2019.103208"},{"key":"ref41","author":"Braun, V","year":"2022","journal-title":"Thematic Analysis : A Practical Guide"},{"key":"ref42","unstructured":"QDAcity2026-08-15https:\/\/qdacity.com\/"},{"key":"ref43","doi-asserted-by":"publisher","DOI":"10.3126\/dsaj.v18i01.67553"},{"key":"ref44","doi-asserted-by":"publisher","DOI":"10.1126\/science.adh2586"},{"key":"ref45","doi-asserted-by":"publisher","DOI":"10.1145\/3803795"},{"key":"ref46","doi-asserted-by":"publisher","DOI":"10.3389\/fdgth.2025.1659134"},{"key":"ref47","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2024.6565"},{"key":"ref48","doi-asserted-by":"publisher","DOI":"10.6092\/issn.2038-3460\/19964"},{"key":"ref49","doi-asserted-by":"publisher","DOI":"10.1007\/s00146-025-02814-9"},{"key":"ref50","doi-asserted-by":"publisher","DOI":"10.3389\/fdgth.2026.1755085"},{"key":"ref51","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2025.34976"},{"key":"ref52","doi-asserted-by":"publisher","DOI":"10.1016\/S0140-6736(26)00204-7"},{"key":"ref53","first-page":"19","author":"Kathleen, LM","year":"2020","journal-title":"Humans and Automated Decision Aids: A Match Made in Heaven?"},{"key":"ref54","unstructured":"ShawSDNaveGThinking--Fast, slow, and artificial: how AI is reshaping human reasoning and the rise of cognitive surrenderThe Wharton School Research Paper20262026-08-20https:\/\/papers.ssrn.com\/sol3\/papers.cfm?abstract_id=6097646"},{"key":"ref55","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-026-42312-6"},{"key":"ref56","first-page":"1","author":"Wampold, B","year":"2015","journal-title":"The Great Psychotherapy Debate: The Evidence for What Makes Psychotherapy Work. 2nd ed"},{"key":"ref57","doi-asserted-by":"publisher","DOI":"10.1080\/10503307.2024.2325510"},{"key":"ref58","doi-asserted-by":"publisher","DOI":"10.1080\/10503307.2015.1119908"},{"key":"ref59","doi-asserted-by":"publisher","DOI":"10.1080\/29974100.2025.2590495"},{"key":"ref60","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-024-03180-7"},{"key":"ref61","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyt.2024.1505024"},{"key":"ref62","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2505.06120"},{"key":"ref63","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2408.09205"},{"key":"ref64","doi-asserted-by":"publisher","DOI":"10.36227\/techrxiv.23589741.v1"},{"key":"ref65","doi-asserted-by":"publisher","DOI":"10.1145\/3589334.3648137"},{"key":"ref66","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2503.24307"},{"key":"ref67","doi-asserted-by":"publisher","DOI":"10.64898\/2026.01.02.25343289"},{"key":"ref68","doi-asserted-by":"publisher","DOI":"10.48550\/arxiv.2311.07264"},{"key":"ref69","doi-asserted-by":"publisher","DOI":"10.3389\/fmed.2024.1359073"},{"key":"ref70","doi-asserted-by":"publisher","DOI":"10.1001\/jamainternmed.2023.1838"},{"key":"ref71","unstructured":"Flaticon2026-08-15https:\/\/www.flaticon.com\/"},{"key":"ref72","unstructured":"Magnific2025-08-15https:\/\/www.magnific.com\/"},{"key":"ref73","unstructured":"SVG Repo2026-08-15https:\/\/svgrepo.com\/"},{"key":"ref74","unstructured":"Icon82026-08-15https:\/\/icons8.com\/"}],"container-title":["JMIR Mental Health"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/mental.jmir.org\/2026\/1\/e96835","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,26]],"date-time":"2026-08-26T15:55:13Z","timestamp":1787759713000},"score":1,"resource":{"primary":{"URL":"https:\/\/mental.jmir.org\/2026\/1\/e96835"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,26]]},"references-count":74,"URL":"https:\/\/doi.org\/10.2196\/96835","relation":{"has-preprint":[{"id-type":"doi","id":"10.2196\/preprints.96835","asserted-by":"object"}]},"ISSN":["2368-7959"],"issn-type":[{"value":"2368-7959","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,26]]}}}