{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,3,18]],"date-time":"2025-03-18T04:13:00Z","timestamp":1742271180492,"version":"3.40.1"},"reference-count":42,"publisher":"Guilford Publications","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Psychodynamic Psychiatry"],"published-print":{"date-parts":[[2025,3]]},"abstract":"<jats:p> Introduction: Psychotherapy researchers often recruit students to code psychotherapy process and outcome variables on individual therapy sessions. To do so, graduate students read verbatim psychotherapy transcripts, listen to audio or watch video of psychotherapy sessions, and then rate process and outcome measures on these sessions. Although prior studies have investigated the reliability and validity of graduate student ratings of personality dysfunction, no current study has investigated the reliability and validity of these ratings made on the basis of psychotherapy transcripts and audio. In this study, we evaluated the degree to which graduate students can reliably and validly code observer-rated personality assessments on the Shedler-Westen Assessment Procedure-200 (SWAP-200). Methods: We related graduate student and experienced clinician-researcher's SWAP-200 scores in an existing dataset of 27 patients undergoing psychoanalytic psychotherapy at early and late phases of treatment. Results: Using truth and bias multilevel models, we found that graduate students tended to underestimate personality pathology at the early phase of treatment and overestimate pathology at the late phase in treatment. These responses resulted in smaller effect sizes in the graduate student ratings, such that patients' personality functioning did not appear to change following treatment. By contrast, the experienced clinician's scores supported moderate to large effect sizes in personality change. Discussion: These deviations from expert judgment may have significant ramifications for evaluating pre and post effect sizes in psychotherapy process studies using graduate student coding. Suggestions for research and practice are discussed. <\/jats:p>","DOI":"10.1521\/pdps.2025.53.1.102","type":"journal-article","created":{"date-parts":[[2025,3,17]],"date-time":"2025-03-17T15:54:22Z","timestamp":1742226862000},"page":"102-120","source":"Crossref","is-referenced-by-count":0,"title":["Can Graduate Students Rate Personality Reliably in Psychoanalytic Treatments Using the Shedler-Westen Assessment Procedure (SWAP-200)?"],"prefix":"10.1521","volume":"53","author":[{"given":"Daniel S.","family":"Spina","sequence":"first","affiliation":[{"name":"Pennsylvania State University, Department of Psychology, University Park, Pennsylvania, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Katie","family":"Aafjes-van Doorn","sequence":"additional","affiliation":[{"name":"Arts and Sciences, New York University, Shanghai, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sarah J.","family":"Horne","sequence":"additional","affiliation":[{"name":"Yeshiva University, Ferkauf Graduate School of Psychology, New York, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Francesco","family":"Gazzillo","sequence":"additional","affiliation":[{"name":"Sapienza University of Rome, Department of Dynamic and Clinical Psychology and Health Studies, Rome, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Federica","family":"Genova","sequence":"additional","affiliation":[{"name":"European University of Rome, Department of Health and Life Sciences, Rome, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bernard S.","family":"Gorman","sequence":"additional","affiliation":[{"name":"Psychoanalytic Research Consortium, New York, USA; Department of Psychology, Hofstra University, New York, USA; Derner Institute, Adelphi University, Garden City, New York, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Karl","family":"Stukenberg","sequence":"additional","affiliation":[{"name":"Xavier University, School of Psychology, Cincinnati, OH, USA; Psychoanalytic Research Consortium, New York, NY, USA; Cincinnati Psychoanalytic Institute, Cincinnati, Ohio, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sherwood","family":"Waldron","sequence":"additional","affiliation":[{"name":"Psychoanalytic Research Consortium, New York, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"377","reference":[{"issue":"4","key":"ref2","doi-asserted-by":"crossref","first-page":"627","DOI":"10.1080\/09515070.2023.2281946","volume":"37","author":"Aafjes-van Doorn K.","year":"2023","journal-title":"Counselling Psychology Quarterly"},{"key":"ref3","volume-title":"Psychodynamic Diagnostic Manual.","author":"Aafjes-van Doorn K.","year":"2025"},{"key":"ref5","volume":"1","author":"Biberdzic M.","year":"2022","journal-title":"Journal of Personality Assessment"},{"issue":"2","key":"ref6","doi-asserted-by":"crossref","first-page":"196","DOI":"10.1037\/pap0000154","volume":"35","author":"Boldrini T.","year":"2018","journal-title":"Psychoanalytic Psychology"},{"issue":"9","key":"ref7","doi-asserted-by":"crossref","first-page":"1202","DOI":"10.1176\/ajp.153.9.1202","volume":"153","author":"Bolton S.","year":"1996","journal-title":"American Journal of Psychiatry"},{"issue":"4","key":"ref8","doi-asserted-by":"crossref","first-page":"284","DOI":"10.1037\/1040-3590.6.4.284","volume":"6","author":"Cicchetti D. V.","year":"1994","journal-title":"Psychological Assessment"},{"key":"ref9","volume-title":"Structured Interview of Personality Organization (STIPO) [Unpublished Manuscript]","author":"Clarkin J. F.","year":"2004"},{"issue":"2","key":"ref10","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1348\/147608305X28727","volume":"78","author":"Cogan R.","year":"2005","journal-title":"Psychology and psychotherapy"},{"key":"ref11","doi-asserted-by":"crossref","first-page":"801","DOI":"10.1037\/a0016596","volume":"77","author":"Connolly-Gibbons M. B.","year":"2009","journal-title":"Journal of Consulting and Clinical Psychology"},{"issue":"3","key":"ref12","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1097\/HRP.0b013e318294f5fd","volume":"21","author":"de Maat S.","year":"2013","journal-title":"Harvard Review of Psychiatry"},{"issue":"4","key":"ref13","first-page":"368","volume":"11","author":"Doss B. D.","year":"2004","journal-title":"Clinical Psychology: Science and Practice"},{"issue":"4","key":"ref14","doi-asserted-by":"crossref","first-page":"561","DOI":"10.1016\/j.psc.2018.07.008","volume":"41","author":"Ellison W. D.","year":"2018","journal-title":"The Psychiatric Clinics of North America"},{"issue":"4","key":"ref15","doi-asserted-by":"crossref","first-page":"627","DOI":"10.1037\/a0019792","volume":"136","author":"Fletcher G. J. O.","year":"2010","journal-title":"Psychological Bulletin"},{"issue":"2","key":"ref16","doi-asserted-by":"crossref","first-page":"184","DOI":"10.1037\/pap0000155","volume":"35","author":"Gazzillo F.","year":"2018","journal-title":"Psychoanalytic Psychology"},{"issue":"2","key":"ref17","doi-asserted-by":"crossref","first-page":"270","DOI":"10.1037\/a0035243","volume":"51","author":"Gazzillo F.","year":"2014","journal-title":"Psychotherapy"},{"issue":"3","key":"ref18","first-page":"556","volume":"58","author":"Gordon A.","year":"2010","journal-title":"Journal of the American Psychoanalytic Association"},{"issue":"2","key":"ref20","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1016\/j.jrp.2006.04.003","volume":"41","author":"Hamaker E. L.","year":"2007","journal-title":"Journal of Research in Personality"},{"key":"ref21","first-page":"84","volume-title":"Handbook of psychotherapy and behavior change","author":"Hill C. E.","year":"2004"},{"issue":"1","key":"ref22","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1146\/annurev.clinpsy.3.022806.091432","volume":"3","author":"Kazdin A. E.","year":"2007","journal-title":"Annual Review of Clinical Psychology"},{"key":"ref23","first-page":"353","volume-title":"Personality disorders and pathology: Integrating clinical assessment and practice in the DSM-5 and ICD-11 era","author":"Levy K. N.","year":"2022"},{"issue":"4","key":"ref24","doi-asserted-by":"crossref","first-page":"376","DOI":"10.1037\/tep0000376","volume":"16","author":"Levy K. N.","year":"2022","journal-title":"Training and Education in Professional Psychology"},{"issue":"2","key":"ref25","doi-asserted-by":"crossref","first-page":"190","DOI":"10.1037\/a0019418","volume":"27","author":"Lingiardi V.","year":"2010","journal-title":"Psychoanalytic Psychology"},{"issue":"1","key":"ref26","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1207\/s15327752jpa8601_04","volume":"86","author":"Lingiardi V.","year":"2006","journal-title":"Journal of Personality Assessment"},{"issue":"1","key":"ref27","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1037\/a0017733","volume":"41","author":"Magnavita J. J.","year":"2010","journal-title":"Professional Psychology: Research and Practice"},{"key":"ref28","volume-title":"The Personality Assessment Inventory professional manual","author":"Morey L.","year":"2017","edition":"1"},{"issue":"3","key":"ref29","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1080\/14733145.2012.739633","volume":"13","author":"Pascual-Leone A.","year":"2013","journal-title":"Counselling and Psychotherapy Research"},{"issue":"6","key":"ref31","doi-asserted-by":"crossref","first-page":"621","DOI":"10.1080\/00223891.2018.1448985","volume":"100","author":"Preti E.","year":"2018","journal-title":"Journal of Personality Assessment"},{"issue":"6","key":"ref32","doi-asserted-by":"crossref","first-page":"571","DOI":"10.1080\/00223891.2018.1475394","volume":"100","author":"Roche M. J.","year":"2018","journal-title":"Journal of Personality Assessment"},{"issue":"6","key":"ref33","doi-asserted-by":"crossref","first-page":"405","DOI":"10.1016\/j.jpsychires.2009.09.011","volume":"44","author":"Ruggero C. J.","year":"2010","journal-title":"Journal of Psychiatric Research"},{"key":"ref34","first-page":"3","volume-title":"Personality disorders","author":"Shedler J.","year":"2022"},{"issue":"1","key":"ref35","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1080\/00223890701357092","volume":"89","author":"Shedler J.","year":"2007","journal-title":"Journal of Personality Assessment"},{"issue":"2","key":"ref36","doi-asserted-by":"crossref","first-page":"420","DOI":"10.1037\/0033-2909.86.2.420","volume":"86","author":"Shrout P. E.","year":"1979","journal-title":"Psychological Bulletin"},{"issue":"6","key":"ref37","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1097\/NMD.0000000000000526","volume":"204","author":"Smith S. W.","year":"2016","journal-title":"Journal of Nervous and Mental Disease"},{"key":"ref38","doi-asserted-by":"crossref","first-page":"363","DOI":"10.1037\/a0024559","volume":"28","author":"Waldron S.","year":"2011","journal-title":"Psychoanalytic Psychology"},{"key":"ref39","doi-asserted-by":"crossref","first-page":"567","DOI":"10.1037\/a0033959","volume":"30","author":"Waldron S.","year":"2013","journal-title":"Psychoanalytic Psychology"},{"issue":"4","key":"ref40","doi-asserted-by":"crossref","first-page":"397","DOI":"10.1080\/10503300802579156","volume":"19","author":"Wasserman R. H.","year":"2009","journal-title":"Psychotherapy Research"},{"issue":"2","key":"ref41","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1037\/a0022936","volume":"118","author":"West T. V.","year":"2011","journal-title":"Psychological Review"},{"issue":"4","key":"ref42","doi-asserted-by":"crossref","first-page":"810","DOI":"10.1037\/0021-843X.116.4.810","volume":"116","author":"Westen D.","year":"2007","journal-title":"Journal of Abnormal Psychology"},{"key":"ref43","doi-asserted-by":"publisher","DOI":"10.1521\/pedi_2012_26_059"},{"key":"ref44","volume-title":"Paradigms of personality assessment","author":"Wiggins J. S.","year":"2005"},{"issue":"10","key":"ref45","doi-asserted-by":"crossref","first-page":"1570","DOI":"10.1176\/ajp.156.10.1570","volume":"156","author":"Zimmerman M.","year":"1999","journal-title":"American Journal of Psychiatry"},{"issue":"4","key":"ref46","doi-asserted-by":"crossref","first-page":"397","DOI":"10.1080\/00223891.2013.852563","volume":"96","author":"Zimmermann J.","year":"2014","journal-title":"Journal of Personality Assessment"}],"container-title":["Psychodynamic Psychiatry"],"original-title":[],"language":"en","deposited":{"date-parts":[[2025,3,17]],"date-time":"2025-03-17T15:55:04Z","timestamp":1742226904000},"score":1,"resource":{"primary":{"URL":"https:\/\/guilfordjournals.com\/doi\/10.1521\/pdps.2025.53.1.102"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3]]},"references-count":42,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025,3]]}},"alternative-id":["10.1521\/pdps.2025.53.1.102"],"URL":"https:\/\/doi.org\/10.1521\/pdps.2025.53.1.102","relation":{},"ISSN":["2162-2590"],"issn-type":[{"value":"2162-2590","type":"print"}],"subject":[],"published":{"date-parts":[[2025,3]]}}}