{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,27]],"date-time":"2026-08-27T07:16:59Z","timestamp":1787815019711,"version":"build-2784847793"},"reference-count":72,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2026,8,27]],"date-time":"2026-08-27T00:00:00Z","timestamp":1787788800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Digit. Health"],"abstract":"<jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Narrative assessment offers unique insight into psychological functioning but is resource-intensive, requiring extensive expert training and time. Recent work suggests Artificial Intelligence (AI), including moderately sized, adaptable large language models (LLMs), can master narrative assessment's complex, multi-step scoring rules. This study examined whether fine-tuning could enable AI raters to accurately assess narratives.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Method<\/jats:title>\n                    <jats:p>We fine-tuned five diverse LLMs (3\u20137 billion parameters) on two datasets, one for the SCORS-G Affective Quality of Representations (AFF) scale and one for Emotional Investment in Relationships (EIR). All narratives were rated by expert human raters. Each dataset was split into training and testing samples to evaluate AI-human agreement.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Fine-tuned models showed good to excellent reliability with human experts for both AFF and EIR. Ensemble ratings, averaged across all five models, yielded excellent single-rater and average-rater ICCs and low error rates, with 97% of AFF and 92% of EIR ratings falling within accepted discrepancy limits.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Fine-tuning effectively adapted AI models into raters capable of reliably scoring complex psychological constructs from narrative. Ensemble-based AI raters can automate AFF and EIR ratings in research settings and, with further validation, may support clinical use via a human-in-the-loop approach. Fine-tuning adaptable AI models offers considerable potential to increase the feasibility and accessibility of sophisticated, multi-method assessment in research and clinical settings alike.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fdgth.2026.1762253","type":"journal-article","created":{"date-parts":[[2026,8,27]],"date-time":"2026-08-27T06:17:30Z","timestamp":1787811450000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["From free association to free parameters: machine-learning empowers artificial intelligence models to rate the affective-relational aspects of narratives like an expert psychologist"],"prefix":"10.3389","volume":"8","author":[{"given":"Caleb J.","family":"Siefert","sequence":"first","affiliation":[{"name":"Department of Behavioral Sciences, University of Michigan-Dearborn","place":["Dearborn, MI, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Barry","family":"Dauphin","sequence":"additional","affiliation":[{"name":"Psychology Department, University of Detroit-Mercy","place":["Detroit, MI, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jenelle","family":"Slavin-Mulford","sequence":"additional","affiliation":[{"name":"Department of Psychological Sciences, Augusta University","place":["Augusta, GA, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sai Dileep Kumar","family":"Mukkamala","sequence":"additional","affiliation":[{"name":"Department of Industrial and Manufacturing Systems Engineering, University of Michigan-Dearborn","place":["Dearborn, MI, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Venkat Akhila","family":"Reddy Tatipally","sequence":"additional","affiliation":[{"name":"Department of Industrial and Manufacturing Systems Engineering, University of Michigan-Dearborn","place":["Dearborn, MI, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Areen","family":"Alsaid","sequence":"additional","affiliation":[{"name":"Department of Industrial and Manufacturing Systems Engineering, University of Michigan-Dearborn","place":["Dearborn, MI, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Abdallah","family":"Chehade","sequence":"additional","affiliation":[{"name":"Department of Industrial and Manufacturing Systems Engineering, University of Michigan-Dearborn","place":["Dearborn, MI, United States"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2026,8,27]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"e15154","DOI":"10.2196\/15154","article-title":"Artificial intelligence and human trust in healthcare: focus on clinicians","volume":"22","author":"Asan","year":"2020","journal-title":"J Med Internet Res"},{"key":"B2","doi-asserted-by":"publisher","first-page":"1317","DOI":"10.1001\/jama.2017.18391","article-title":"Big data and machine learning in health care","volume":"319","author":"Beam","year":"2018","journal-title":"JAMA"},{"key":"B3","doi-asserted-by":"publisher","first-page":"44","DOI":"10.1038\/s41591-018-0300-7","article-title":"High-performance medicine: the convergence of human and artificial intelligence","volume":"25","author":"Topol","year":"2019","journal-title":"Nat Med"},{"key":"B4","doi-asserted-by":"publisher","first-page":"160035","DOI":"10.1038\/sdata.2016.35","article-title":"MIMIC-III, a freely accessible critical care database","volume":"3","author":"Johnson","year":"2016","journal-title":"Sci Data"},{"key":"B5","doi-asserted-by":"publisher","first-page":"2199","DOI":"10.1001\/jama.2018.17163","article-title":"Clinical decision support in the era of artificial intelligence","volume":"320","author":"Shortliffe","year":"2018","journal-title":"JAMA"},{"key":"B6","doi-asserted-by":"publisher","first-page":"116","DOI":"10.1007\/s11920-019-1094-0","article-title":"Artificial intelligence for mental health and mental illnesses: an overview","volume":"21","author":"Graham","year":"2019","journal-title":"Curr Psychiatry Rep"},{"key":"B7","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1016\/j.apnu.2019.08.001","article-title":"Mental health delivery in primary care: the perspectives of primary care providers","volume":"33","author":"Poghosyan","year":"2019","journal-title":"Arch Psychiatr Nurs"},{"key":"B8","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1016\/j.brat.2016.06.004","article-title":"Addressing the treatment gap: a key challenge for extending evidence-based psychosocial interventions","volume":"88","author":"Kazdin","year":"2017","journal-title":"Behav Res Ther"},{"key":"B9","doi-asserted-by":"publisher","first-page":"128","DOI":"10.1037\/0003-066X.56.2.128","article-title":"Psychological testing and psychological assessment: a review of evidence and issues","volume":"56","author":"Meyer","year":"2001","journal-title":"Am Psychol"},{"key":"B10","doi-asserted-by":"publisher","first-page":"2515","DOI":"10.1056\/NEJMsa043266","article-title":"Prevalence and treatment of mental disorders, 1990 to 2003","volume":"352","author":"Kessler","year":"2005","journal-title":"N Engl J Med"},{"key":"B11","doi-asserted-by":"publisher","first-page":"e1097","DOI":"10.1542\/peds.2009-2712","article-title":"Evaluation of the PHQ-2 as a brief screen for detecting major depression among adolescents","volume":"125","author":"Richardson","year":"2010","journal-title":"Pediatrics"},{"key":"B12","doi-asserted-by":"publisher","first-page":"507","DOI":"10.1177\/1073191116649658","article-title":"Implementing dynamic assessments in psychotherapy","volume":"23","author":"Hopwood","year":"2016","journal-title":"Assessment"},{"key":"B13","volume-title":"Multimethod Clinical Assessment","author":"Hopwood","year":"2014"},{"key":"B14","volume-title":"The Social Cognition and Object Relations Scale-Global Rating Method (SCORS-G): A Comprehensive Guide for Clinicians and Researchers","author":"Stein","year":"2018"},{"key":"B15","doi-asserted-by":"publisher","first-page":"435","DOI":"10.1080\/00223891.2016.1236343","article-title":"Evidence-based psychological assessment","volume":"99","author":"Bornstein","year":"2017","journal-title":"J Pers Assess"},{"key":"B16","doi-asserted-by":"publisher","first-page":"607","DOI":"10.1080\/00223891.2012.684118","article-title":"An inter-rater reliability study for the rorschach performance assessment system","volume":"94","author":"Viglione","year":"2012","journal-title":"J Pers Assess"},{"key":"B17","doi-asserted-by":"crossref","first-page":"15","DOI":"10.4135\/9781506335117.n2","article-title":"Exploring psychological themes through life-narrative accounts","volume-title":"Varieties of Narrative Analysis","author":"McAdams","year":"2012"},{"key":"B18","doi-asserted-by":"publisher","first-page":"519","DOI":"10.1177\/1948550617698202","article-title":"Research methods for studying narrative identity: a primer","volume":"8","author":"Adler","year":"2017","journal-title":"Soc Psychol Personal Sci"},{"key":"B19","doi-asserted-by":"publisher","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","article-title":"BioBERT: a pre-trained biomedical language representation model for biomedical text mining","volume":"36","author":"Lee","year":"2020","journal-title":"Bioinformatics"},{"key":"B20","doi-asserted-by":"publisher","first-page":"420","DOI":"10.3390\/bioengineering10040420","article-title":"A disease-prediction protocol integrating triage priority and BERT-based transfer learning for intelligent triage","volume":"10","author":"Wang","year":"2023","journal-title":"Bioengineering"},{"key":"B21","doi-asserted-by":"publisher","first-page":"848","DOI":"10.1186\/s40001-025-03196-w","article-title":"Artificial intelligence in healthcare and medicine: clinical applications, therapeutic advances, and future perspectives","volume":"30","author":"Fahim","year":"2025","journal-title":"Eur J Med Res"},{"key":"B22","doi-asserted-by":"publisher","first-page":"e15360","DOI":"10.2196\/15360","article-title":"The personalization of conversational agents in health care: systematic review","volume":"21","author":"Kocaballi","year":"2019","journal-title":"J Med Internet Res"},{"key":"B23","doi-asserted-by":"publisher","first-page":"457","DOI":"10.1093\/jamia\/ocz200","article-title":"Deep learning in clinical natural language processing: a methodical review","volume":"27","author":"Wu","year":"2020","journal-title":"J Am Med Inform Assoc"},{"key":"B24","doi-asserted-by":"publisher","first-page":"301","DOI":"10.3390\/info16040301","article-title":"nBERT: harnessing NLP for emotion recognition in psychotherapy to transform mental health care","volume":"16","author":"Rasool","year":"2025","journal-title":"Information"},{"key":"B25","doi-asserted-by":"publisher","first-page":"56","DOI":"10.3390\/ai6030056","article-title":"Emotion-aware embedding fusion in large language models (flan-T5, llama 2, DeepSeek-R1, and ChatGPT 4) for intelligent response generation","volume":"6","author":"Rasool","year":"2025","journal-title":"AI"},{"key":"B26","volume-title":"Social Cognition and Object Relations Scale: Global Rating Method (SCORS-G; 4th ed)","author":"Stein","year":"2011"},{"key":"B27","doi-asserted-by":"publisher","first-page":"429","DOI":"10.1037\/0033-2909.109.3.429","article-title":"Social cognition and object relations","volume":"109","author":"Westen","year":"1991","journal-title":"Psychol Bull"},{"key":"B28","doi-asserted-by":"publisher","first-page":"533","DOI":"10.1080\/00223891.2012.670680","article-title":"Exploring the construct validity of the social cognition and object relations scale in a clinical sample","volume":"94","author":"Stein","year":"2012","journal-title":"J Pers Assess"},{"key":"B29","first-page":"345","article-title":"The social cognition and object relations scale-global rating method (SCORS-G) in clinical practice","volume-title":"Comprehensive Handbook of Psychological Assessment: Vol. 2. Personality Assessment","author":"Cain","year":"2020"},{"key":"B30","doi-asserted-by":"publisher","first-page":"201","DOI":"10.1521\/pedi_2021_35_535","article-title":"Exploring the alternative model for personality disorders using SCORS-G ratings on thematic apperception test narratives","volume":"36","author":"Ruchensky","year":"2022","journal-title":"J Pers Disord"},{"key":"B31","doi-asserted-by":"publisher","first-page":"606","DOI":"10.1037\/a0032085","article-title":"Development and preliminary validation of the level of care Index (LOCI) from the personality assessment inventory (PAI) in a psychiatric sample","volume":"25","author":"Sinclair","year":"2013","journal-title":"Psychol Assess"},{"key":"B32","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1080\/00223891.2017.1418747","article-title":"Flexibility and rigidity in object relational functioning: assessing change in suicidal ideation and global psychiatric functioning using the SCORS-G","volume":"100","author":"Clemence","year":"2018","journal-title":"J Pers Assess"},{"key":"B33","first-page":"365","article-title":"Exploring the inner world of self-mutilating borderline patients: a rorschach investigation","volume":"64","author":"Fowler","year":"2000","journal-title":"Bull Menninger Clin"},{"key":"B34","doi-asserted-by":"publisher","first-page":"145","DOI":"10.1521\/pedi_2019_33_435","article-title":"Quality of internalized object representations and suicidality in individuals with anaclitic and introjective personality styles","volume":"35","author":"Lewis","year":"2021","journal-title":"J Pers Disord"},{"key":"B35","doi-asserted-by":"publisher","first-page":"123","DOI":"10.1002\/cpp.1826","article-title":"Relationship between patient pre-treatment object relations functioning and psychodynamic techniques early in treatment","volume":"21","author":"Mullin","year":"2014","journal-title":"Clin Psychol Psychother"},{"key":"B36","doi-asserted-by":"publisher","first-page":"501","DOI":"10.1002\/cpp.2021","article-title":"Changes in object relations over the course of psychodynamic psychotherapy","volume":"24","author":"Mullin","year":"2017","journal-title":"Clin Psychol Psychother"},{"key":"B37","doi-asserted-by":"publisher","first-page":"978","DOI":"10.1002\/cpp.2552","article-title":"Object relations assessment predicts outpatients' Attrition, early working alliance and psychotherapy process","volume":"28","author":"Vaughn","year":"2021","journal-title":"Clin Psychol Psychother"},{"key":"B38","doi-asserted-by":"publisher","first-page":"418","DOI":"10.1037\/a0029566","article-title":"Quality of object representations related to service utilization in a long-term residential treatment center","volume":"49","author":"Fowler","year":"2012","journal-title":"Psychotherapy"},{"key":"B39","doi-asserted-by":"publisher","first-page":"306","DOI":"10.1207\/s15327752jpa8303_12","article-title":"Personality and symptom change in treatment-refractory inpatients: evaluation of the phase model of change using rorschach, TAT, and DSM-IV axis V","volume":"83","author":"Fowler","year":"2004","journal-title":"J Pers Assess"},{"key":"B40","doi-asserted-by":"publisher","first-page":"507","DOI":"10.1097\/NMD.0000000000001799","article-title":"The role of object relations in the physician-patient relationship in a primary care setting","volume":"212","author":"Porcerelli","year":"2024","journal-title":"J Nerv Ment Dis"},{"key":"B41","doi-asserted-by":"publisher","first-page":"156","DOI":"10.1080\/00223891.2017.1369984","article-title":"The use of the social cognition and object relations scale in a primary care setting","volume":"100","author":"Richardson","year":"2018","journal-title":"J Pers Assess"},{"key":"B42","doi-asserted-by":"publisher","first-page":"314","DOI":"10.1037\/a0036286","article-title":"Object relations, interpersonal functioning, and health in a nonclinical sample: construct validation and norms for the TAT SCORS-G","volume":"31","author":"Bram","year":"2014","journal-title":"Psychoanal Psychol"},{"key":"B43","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1037\/pap0000446","article-title":"The utility of the self\u2013other narrative evaluation scale (SONES) for assessing internal representations in interpersonal narratives","volume":"40","author":"Siefert","year":"2023","journal-title":"Psychoanal Psychol"},{"key":"B44","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1177\/25152459251343582","article-title":"Large language models for psychological assessment: a comprehensive overview","volume":"8","author":"Brickman","year":"2025","journal-title":"Adv Methods Pract Psychol Sci"},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.1093\/pnasnexus\/pgae231","article-title":"Large language models can infer psychological dispositions of social media users","volume":"3","author":"Peters","year":"2024","journal-title":"PNAS Nexus"},{"key":"B46","author":"Simons","year":"2024"},{"key":"B47","doi-asserted-by":"publisher","first-page":"541","DOI":"10.1038\/s41562-025-02389-x","article-title":"Assessing personality using zero-shot generative AI scoring of brief open-ended text","volume":"10","author":"Wright","year":"2026","journal-title":"Nat Hum Behav"},{"key":"B48","doi-asserted-by":"publisher","first-page":"1398885","DOI":"10.3389\/frai.2025.1398885","article-title":"From llama to language: prompt-engineering allows general-purpose artificial intelligence to rate narratives like expert psychologists","volume":"8","author":"Dauphin","year":"2025","journal-title":"Front Artif Intell."},{"key":"B49","doi-asserted-by":"publisher","first-page":"164","DOI":"10.1037\/pap0000573","article-title":"Rater: The utility of Artificial Intelligence bots for assigning narrative ratings with four dimensions of the Social Cognition and Object Relations Scale-Global Rating Method","volume":"43","author":"Siefert","year":"2026","journal-title":"Psychoanal Psychol"},{"key":"B50","author":"Na","year":"2026"},{"key":"B51","doi-asserted-by":"publisher","first-page":"122","DOI":"10.1037\/abn0001047","article-title":"Language-based AI modeling of personality traits and pathology from life narrative interviews","volume":"135","author":"Oltmanns","year":"2026","journal-title":"J Psychopathol Clin Sci"},{"key":"B52","author":"Galatzer-Levy","year":"2023"},{"key":"B53","first-page":"164","article-title":"AffectLLM: a large language model for rating the affective quality of representations","author":"Chehade","year":"2025"},{"key":"B54","doi-asserted-by":"publisher","first-page":"122","DOI":"10.1080\/00223891.2017.1336716","article-title":"Exploring the factor structure of the social cognition and object relations\u2013global rating method: support for two- and three-factor models","volume":"100","author":"Siefert","year":"2018","journal-title":"J Pers Assess"},{"key":"B55","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2309.11235","article-title":"Openchat: advancing open-source language models with mixed-quality data","author":"Wang","year":"2023","journal-title":"arXiv:2309.11235"},{"key":"B56","year":"2024"},{"key":"B57","author":"Jiang","year":"2023"},{"key":"B58","author":"Yang","year":"2025"},{"key":"B59","year":"2024"},{"key":"B60","volume-title":"Thematic Apperception Test","author":"Murray","year":"1943"},{"key":"B61","doi-asserted-by":"publisher","first-page":"374","DOI":"10.1080\/00223891.2024.2425660","article-title":"Moving toward an online thematic apperception test (TAT): the impact of administration modifications on narrative length and story richness","volume":"107","author":"Slavin-Mulford","year":"2025","journal-title":"J Pers Assess"},{"key":"B62","doi-asserted-by":"publisher","first-page":"469","DOI":"10.1080\/00223891.2023.2286459","article-title":"A comparison of different thematic apperception test administration methods on narrative length and story richness as measured by the social cognition and object relations scale-global rating method (SCORS-G)","volume":"106","author":"Coleman","year":"2024","journal-title":"J Pers Assess"},{"key":"B63","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511527937","volume-title":"Motivation and Personality: Handbook of Thematic Content Analysis","author":"Smith","year":"1992"},{"key":"B64","doi-asserted-by":"publisher","first-page":"10088","DOI":"10.52202\/075280-0441","article-title":"QLoRA: efficient finetuning of quantized LLMs","volume":"36","author":"Dettmers","year":"2023","journal-title":"Adv Neural Inf Process Syst"},{"key":"B65","volume-title":"arXiv:2106.09685","author":"Hu","year":"2021"},{"key":"B66","doi-asserted-by":"publisher","first-page":"e75030","DOI":"10.2196\/75030","article-title":"Aiding large language models using clinical scoresheets for neurobehavioral diagnostic classification from text: algorithm development and validation","volume":"4","author":"Lin","year":"2025","journal-title":"JMIR AI"},{"key":"B67","doi-asserted-by":"publisher","first-page":"284","DOI":"10.1037\/1040-3590.6.4.284","article-title":"Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology","volume":"6","author":"Cicchetti","year":"1994","journal-title":"Psychol Assess"},{"key":"B68","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1016\/j.jcm.2016.02.012","article-title":"A guideline of selecting and reporting intraclass correlation coefficients for reliability research","volume":"15","author":"Koo","year":"2016","journal-title":"J Chiropr Med"},{"key":"B69","doi-asserted-by":"publisher","first-page":"597","DOI":"10.22237\/jmasm\/1257035100","article-title":"New effect size rules of thumb","volume":"8","author":"Sawilowsky","year":"2009","journal-title":"J Mod Appl Stat Methods"},{"key":"B70","doi-asserted-by":"publisher","first-page":"235","DOI":"10.1037\/pas0001437","article-title":"The social cognition and object relations scale is reliable, valid, and ethical in clinical practice and research: comment on Sinclair et al. (2023)","volume":"38","author":"Stein","year":"2026","journal-title":"Psychol Assess"},{"key":"B71","doi-asserted-by":"publisher","first-page":"438","DOI":"10.1037\/cou0000382","article-title":"Machine learning and natural language processing in psychotherapy research: alliance as example use case","volume":"67","author":"Goldberg","year":"2020","journal-title":"J Couns Psychol"},{"key":"B72","doi-asserted-by":"publisher","first-page":"e53562","DOI":"10.2196\/53562","article-title":"Automated behavioral coding to enhance the effectiveness of motivational interviewing in a chat-based suicide prevention helpline: secondary analysis of a clinical trial","volume":"26","author":"Pellemans","year":"2024","journal-title":"J Med Internet Res"}],"container-title":["Frontiers in Digital Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2026.1762253\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,8,27]],"date-time":"2026-08-27T06:17:33Z","timestamp":1787811453000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2026.1762253\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,27]]},"references-count":72,"alternative-id":["10.3389\/fdgth.2026.1762253"],"URL":"https:\/\/doi.org\/10.3389\/fdgth.2026.1762253","relation":{},"ISSN":["2673-253X"],"issn-type":[{"value":"2673-253X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,27]]},"article-number":"1762253"}}