{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,2]],"date-time":"2026-10-02T22:17:38Z","timestamp":1790979458399,"version":"4.1.0"},"reference-count":44,"publisher":"Public Library of Science (PLoS)","issue":"2","license":[{"start":{"date-parts":[[2025,2,12]],"date-time":"2025-02-12T00:00:00Z","timestamp":1739318400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.plosmentalhealth.org"],"crossmark-restriction":false},"short-container-title":["PLOS Ment Health"],"abstract":"<jats:p>\u201cCan machines be therapists?\u201d is a question receiving increased attention given the relative ease of working with generative artificial intelligence. Although recent (and decades-old) research has found that humans struggle to tell the difference between responses from machines and humans, recent findings suggest that artificial intelligence can write empathically and the generated content is rated highly by therapists and outperforms professionals. It is uncertain whether, in a preregistered competition where therapists and ChatGPT respond to therapeutic vignettes about couple therapy, a) a panel of participants can tell which responses are ChatGPT-generated and which are written by therapists (N = 13), b) the generated responses or the therapist-written responses fall more in line with key therapy principles, and c) linguistic differences between conditions are present. In a large sample (N = 830), we showed that a) participants could rarely tell the difference between responses written by ChatGPT and responses written by a therapist, b) the responses written by ChatGPT were generally rated higher in key psychotherapy principles, and c) the language patterns between ChatGPT and therapists were different. Using different measures, we then confirmed that responses written by ChatGPT were rated higher than the therapist\u2019s responses suggesting these differences may be explained by part-of-speech and response sentiment. This may be an early indication that ChatGPT has the potential to improve psychotherapeutic processes. We anticipate that this work may lead to the development of different methods of testing and creating psychotherapeutic interventions. Further, we discuss limitations (including the lack of the therapeutic context), and how continued research in this area may lead to improved efficacy of psychotherapeutic interventions allowing such interventions to be placed in the hands of individuals who need them the most.<\/jats:p>","DOI":"10.1371\/journal.pmen.0000145","type":"journal-article","created":{"date-parts":[[2025,2,12]],"date-time":"2025-02-12T13:23:56Z","timestamp":1739366636000},"page":"e0000145","update-policy":"https:\/\/doi.org\/10.1371\/journal.pmen.corrections_policy","source":"Crossref","is-referenced-by-count":69,"title":["When ELIZA meets therapists: A Turing test for the heart and mind"],"prefix":"10.1371","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1122-6537","authenticated-orcid":true,"given":"S. Gabe","family":"Hatch","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zachary T.","family":"Goodman","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Laura","family":"Vowels","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6083-4179","authenticated-orcid":true,"given":"H. Dorian","family":"Hatch","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alyssa L.","family":"Brown","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shayna","family":"Guttman","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yunying","family":"Le","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5700-6751","authenticated-orcid":true,"given":"Benjamin","family":"Bailey","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7716-1756","authenticated-orcid":true,"given":"Russell J.","family":"Bailey","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Charlotte R.","family":"Esplin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9022-1899","authenticated-orcid":true,"given":"Steven M.","family":"Harris","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"D. Payton","family":"Holt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4191-7865","authenticated-orcid":true,"given":"Merranda","family":"McLaughlin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Patrick","family":"O\u2019Connell","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Karen","family":"Rothman","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lane","family":"Ritchie","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0228-7995","authenticated-orcid":true,"given":"D. Nicholas","family":"Top","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Scott R.","family":"Braithwaite","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2025,2,12]]},"reference":[{"issue":"236","key":"pmen.0000145.ref001","first-page":"433","article-title":"Computing machinery and intelligence","volume":"59","author":"A. Turing","year":"1950","journal-title":"Mind A Q Rev Psychol Philos"},{"issue":"1","key":"pmen.0000145.ref002","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1145\/365153.365168","article-title":"ELIZA\u2014a computer program for the study of natural language communication between man and machine","volume":"9","author":"J. Weizenbaum","year":"1966","journal-title":"Commun ACM"},{"key":"pmen.0000145.ref003","doi-asserted-by":"crossref","first-page":"101753","DOI":"10.1016\/j.artmed.2019.101753","article-title":"Artificial intelligence and the future of psychiatry: Insights from a global physician survey","volume":"102","author":"P Doraiswamy","year":"2020","journal-title":"Artif Intell Med"},{"issue":"1","key":"pmen.0000145.ref004","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1038\/s42256-022-00593-2","article-title":"Human\u2013AI collaboration enables more empathic conversations in text-based peer-to-peer mental health support","volume":"5","author":"A Sharma","year":"2023","journal-title":"Nat Mach Intell"},{"key":"pmen.0000145.ref005","article-title":"Thanks a Bot: Leveraging Artificial Intelligence for Improved Workplace Appreciation.","author":"S Hatch","journal-title":"OSF Preprint."},{"issue":"6","key":"pmen.0000145.ref006","doi-asserted-by":"crossref","first-page":"589","DOI":"10.1001\/jamainternmed.2023.1838","article-title":"Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum","volume":"183","author":"J Ayers","year":"2023","journal-title":"JAMA Intern Med"},{"issue":"2","key":"pmen.0000145.ref007","first-page":"100077","article-title":"Are chatbots the new relationship experts? Insights from three studies. Computers in Human Behavior","volume":"2","author":"L Vowels","year":"2024","journal-title":"Artificial Humans"},{"issue":"2","key":"pmen.0000145.ref008","doi-asserted-by":"crossref","first-page":"100078","DOI":"10.1016\/j.chbah.2024.100078","article-title":"AI in relationship counselling: Evaluating ChatGPT\u2019s therapeutic efficacy in providing relationship advice.","volume":"2","author":"L Vowels","year":"2024","journal-title":"Computers in Human Behavior: Artificial Humans"},{"issue":"5","key":"pmen.0000145.ref009","doi-asserted-by":"crossref","first-page":"609","DOI":"10.1521\/jscp.2007.26.5.609","article-title":"ePREP: Computer based prevention of relationship dysfunction, depression and anxiety.","volume":"26","author":"S Braithwaite","year":"2007","journal-title":"J Soc Clin Psychol"},{"issue":"1","key":"pmen.0000145.ref010","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1037\/a0014061","article-title":"A randomized clinical trial of a computer based preventive intervention: Replication and extension of ePREP","volume":"23","author":"S Braithwaite","year":"2009","journal-title":"J Fam Psychol"},{"issue":"2","key":"pmen.0000145.ref011","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1016\/j.brat.2010.11.002","article-title":"Computer-based dissemination: A randomized clinical trial of ePREP using the actor partner interdependence model","volume":"49","author":"S Braithwaite","year":"2011","journal-title":"Behav Res Ther"},{"issue":"1","key":"pmen.0000145.ref012","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1111\/famp.12020","article-title":"Translation of Integrative Behavioral Couple Therapy to a web-based intervention","volume":"52","author":"B Doss","year":"2013","journal-title":"Fam Process"},{"issue":"4","key":"pmen.0000145.ref013","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1037\/ccp0000479","article-title":"Online programs improve relationship functioning for distressed low-income couples: Results from a nationwide randomized controlled trial","volume":"88","author":"B Doss","year":"2020","journal-title":"J Consult Clin Psychol"},{"issue":"3","key":"pmen.0000145.ref014","doi-asserted-by":"crossref","first-page":"351","DOI":"10.1177\/1066480718786491","article-title":"Marital text: A feasibility study","volume":"26","author":"S Hatch","year":"2018","journal-title":"Fam J"},{"issue":"2","key":"pmen.0000145.ref015","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1080\/15332691.2019.1667936","article-title":"Texts 4 romantic relationships\u2013a randomized controlled trial","volume":"19","author":"S Hatch","year":"2020","journal-title":"J Couple Relatsh Ther"},{"issue":"4","key":"pmen.0000145.ref016","doi-asserted-by":"crossref","first-page":"296","DOI":"10.1037\/a0027743","article-title":"Healthy marriage initiatives: On the need for empiricism in policy implementation","volume":"67","author":"M. Johnson","year":"2012","journal-title":"Am Psychol."},{"issue":"2","key":"pmen.0000145.ref017","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1037\/a0031793","article-title":"Optimistic or quixotic? More data on marriage and relationship education programs for lower income couples","volume":"68","author":"M. Johnson","year":"2013","journal-title":"Am Psychol."},{"key":"pmen.0000145.ref018","doi-asserted-by":"crossref","first-page":"100661","DOI":"10.1016\/j.invent.2023.100661","article-title":"A randomized controlled trial of the online OurRelationship program with varying levels of coach support","volume":"34","author":"Y Le","year":"2023","journal-title":"Internet Interv"},{"issue":"1","key":"pmen.0000145.ref019","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1111\/famp.12548","article-title":"Heteronormative relationship education for same-gender couples","volume":"60","author":"S Hatch","year":"2021","journal-title":"Fam Process"},{"issue":"3","key":"pmen.0000145.ref020","doi-asserted-by":"crossref","first-page":"1045","DOI":"10.1111\/famp.12710","article-title":"Online relationship education for help-seeking low-income couples: A Bayesian replication and extension of the OurRelationship and ePREP programs","volume":"61","author":"S Hatch","year":"2022","journal-title":"Fam Process"},{"key":"pmen.0000145.ref021","article-title":"Leveraging Machine Learning to Optimize the Amount and Type of Practitioner Contact in Online Relationship Education for Low-Income Couples [Doctoral dissertation, University of Miami].","author":"S. Hatch","year":"2023"},{"issue":"3","key":"pmen.0000145.ref022","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1111\/j.1741-3737.2012.00977.x","article-title":"What\u2019s (not) wrong with low-income marriages","volume":"74","author":"T Trail","year":"2012","journal-title":"J Marriage Fam"},{"key":"pmen.0000145.ref023","author":"American Psychological Association.","year":"2016","journal-title":"Ethical principles of psychologists and code of conduct"},{"key":"pmen.0000145.ref024","unstructured":"OpenAI. Models Documentation. 2023. Available from: https:\/\/platform.openai.com\/docs\/models\/gpt-4-and-gpt-4-turbo."},{"issue":"3","key":"pmen.0000145.ref025","doi-asserted-by":"crossref","first-page":"666","DOI":"10.1016\/j.beth.2012.01.001","article-title":"Unified protocol for transdiagnostic treatment of emotional disorders: a randomized controlled trial","volume":"43","author":"T Farchione","year":"2012","journal-title":"Behav Ther"},{"issue":"2","key":"pmen.0000145.ref026","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1037\/0022-006X.72.2.176","article-title":"Traditional versus integrative behavioral couple therapy for significantly and chronically distressed married couples.","volume":"72","author":"A Christensen","year":"2004","journal-title":"J Consult Clin Psychol"},{"issue":"4","key":"pmen.0000145.ref027","doi-asserted-by":"crossref","first-page":"467","DOI":"10.1037\/a0034332","article-title":"Expanding the lens of evidence-based practice in psychotherapy: A common factors perspective","volume":"51","author":"K Laska","year":"2014","journal-title":"Psychother"},{"issue":"3","key":"pmen.0000145.ref028","doi-asserted-by":"crossref","first-page":"270","DOI":"10.1002\/wps.20238","article-title":"How important are the common factors in psychotherapy? An update","volume":"14","author":"B. Wampold","year":"2015","journal-title":"World Psychiatry"},{"key":"pmen.0000145.ref029","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1037\/pri0000100","article-title":"Common Factors as a therapeutic approach: What is required","volume":"4","author":"R Bailey","year":"2019","journal-title":"Practice Innovations"},{"key":"pmen.0000145.ref030","doi-asserted-by":"crossref","DOI":"10.1037\/0000343-000","volume-title":"Common factors therapy: A principle-based treatment framework","author":"R Bailey","year":"2023"},{"key":"pmen.0000145.ref031","volume-title":"Persuasion and Healing: A comparative study of psychotherapy","author":"J Frank","year":"1991","edition":"3"},{"issue":"3","key":"pmen.0000145.ref032","doi-asserted-by":"crossref","first-page":"412","DOI":"10.1111\/j.1939-0025.1936.tb05248.x","article-title":"Some implicit common factors in diverse methods of psychotherapy","volume":"6","author":"S. Rosenzweig","year":"1936","journal-title":"American Journal of Orthopsychiatry"},{"key":"pmen.0000145.ref033","doi-asserted-by":"crossref","DOI":"10.4324\/9780203582015","volume-title":"The great psychotherapy debate: The evidence for what makes psychotherapy work","author":"B Wampold","year":"2015","edition":"2"},{"issue":"4","key":"pmen.0000145.ref034","doi-asserted-by":"crossref","first-page":"747","DOI":"10.1037\/bul0000015","article-title":"The effects of cognitive behavioral therapy as an anti-depressive treatment is falling: A meta-analysis","volume":"141","author":"T Johnsen","year":"2015","journal-title":"Psychol Bull"},{"issue":"3","key":"pmen.0000145.ref035","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1037\/bul0000055","article-title":"The effects of cognitive-behavioral therapy for depression are not falling: A re-analysis of Johnsen and Friborg (2015).","volume":"143","author":"B Lj\u00f3tsson","year":"2017","journal-title":"Psychol Bull"},{"key":"pmen.0000145.ref036","doi-asserted-by":"crossref","article-title":"Culturally informed therapy for Schizophrenia: A family-focused cognitive behavioral approach, clinician guide","author":"A Weisman de Mamani","DOI":"10.1093\/med-psych\/9780197500644.001.0001"},{"issue":"4","key":"pmen.0000145.ref037","doi-asserted-by":"crossref","first-page":"356","DOI":"10.1080\/13651501.2020.1800043","article-title":"Risk of suicide after discharge from inpatient psychiatric care: a systematic review.","volume":"25","author":"P O\u2019Connell","year":"2021","journal-title":"International journal of psychiatry in clinical practice"},{"key":"pmen.0000145.ref038","author":"E Loper","year":"2002","journal-title":"Nltk: The natural language toolkit"},{"key":"pmen.0000145.ref039","article-title":"Decoding Emotion Dynamics in Videos using Dynamic Exploratory Graph Analysis and Zero-Shot Image Classification: A Simulation and Tutorial using the transforEmotion R package.","author":"A Toma\u0161evi\u0107","year":"2024","journal-title":"PsyArXiv Preprint"},{"key":"pmen.0000145.ref040","first-page":"4290","article-title":"UDPipe: trainable pipeline for processing CoNLL-U files performing tokenization, morphological analysis, pos tagging and parsing","author":"M Straka","year":"2016","journal-title":"In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC\u201916)"},{"key":"pmen.0000145.ref041","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1016\/j.brat.2016.12.016","article-title":"An introduction to using Bayesian linear regression with clinical data.","volume":"98","author":"S Baldwin","year":"2017","journal-title":"Behav Res Ther"},{"key":"pmen.0000145.ref042","first-page":"2","article-title":"Bayesian first aid: A package that implements Bayesian alternatives to the classical*. test functions in R.","author":"R. B\u00e5\u00e5th","year":"2014","journal-title":"In Proceedings of useR"},{"key":"pmen.0000145.ref043","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v080.i01","article-title":"brms: An R package for Bayesian multilevel models using Stan","volume":"80","author":"P. B\u00fcrkner","year":"2017","journal-title":"J Stat Softw"},{"key":"pmen.0000145.ref044","article-title":"Prompt engineering and its usability to improve modern psychology chatbots.","author":"I Nordgren","year":"2023"}],"updated-by":[{"DOI":"10.1371\/journal.pmen.0000426","type":"correction","label":"Correction","source":"publisher","updated":{"date-parts":[[2025,8,29]],"date-time":"2025-08-29T00:00:00Z","timestamp":1756425600000}}],"container-title":["PLOS Mental Health"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pmen.0000145","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,12]],"date-time":"2025-02-12T13:24:10Z","timestamp":1739366650000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pmen.0000145"}},"subtitle":[],"editor":[{"given":"Brian","family":"Bauer","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2025,2,12]]},"references-count":44,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2025,2,12]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pmen.0000145","relation":{},"ISSN":["2837-8156"],"issn-type":[{"value":"2837-8156","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,12]]}}}