{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,30]],"date-time":"2026-07-30T16:01:05Z","timestamp":1785427265217,"version":"3.56.0"},"reference-count":27,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T00:00:00Z","timestamp":1781654400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62372494"],"award-info":[{"award-number":["62372494"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"award":["62372494"],"award-info":[{"award-number":["62372494"]}],"id":[{"id":"https:\/\/ror.org\/01h0zpd94","id-type":"ROR","asserted-by":"publisher"}]},{"name":"Guangdong Key Disciplines Project","award":["2022ZDJS139"],"award-info":[{"award-number":["2022ZDJS139"]}]},{"name":"Zhuhai Industry\u2013University Research Project","award":["2220004002671"],"award-info":[{"award-number":["2220004002671"]}]},{"award":["2220004002671"],"award-info":[{"award-number":["2220004002671"]}],"id":[{"id":"https:\/\/ror.org\/058643s73","id-type":"ROR","asserted-by":"publisher"}]},{"name":"Engineering Technology Research Center of Guangdong Provincial Key Scientific Research Platform Project","award":["2024GCZX001"],"award-info":[{"award-number":["2024GCZX001"]}]},{"name":"National Undergraduate Training Program on Innovation and Entrepreneurship","award":["202313684003S"],"award-info":[{"award-number":["202313684003S"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Session dialogue assessment based on machine learning is gradually becoming an effective solution for therapeutic alliance measurement which is an important factor for successful psychotherapy. However, most existing models assume clean and pre-structured dialogue transcripts, whereas real-world counseling documentation often contains heterogeneous case reports. This gap limits the applicability of current automated assessment models in realistic documentation scenarios. In this work, we propose a framework for automated working alliance assessment from complex, multilingual reports. First, language-specific BERT models are fine-tuned to process case reports across different languages, enabling accurate speaker role delineation and dialogue structuring. Second, Gemini-2.5-Flash is leveraged to annotate the dialogues with working alliance ratings. Third, a hybrid feature representation strategy is then developed to jointly capture linguistic style and semantic content from the counseling dialogues. Furthermore, an entropy-based mutual information analysis is conducted to identify the most informative linguistic features. Finally, the extracted hybrid features serve as inputs to XGBoost for alliance assessment. In experiments, the proposed framework shows better performance in the comparison with SOTA methods and generalization ability.<\/jats:p>","DOI":"10.3390\/e28060699","type":"journal-article","created":{"date-parts":[[2026,6,18]],"date-time":"2026-06-18T00:49:42Z","timestamp":1781743782000},"page":"699","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Automated Working Alliance Assessment in Psychological Counseling Using Gemini and XGBoost"],"prefix":"10.3390","volume":"28","author":[{"given":"Yuexi","family":"Li","sequence":"first","affiliation":[{"name":"College of Computer Science and Technology, Jilin University, Changchun 130012, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ningtao","family":"Sun","sequence":"additional","affiliation":[{"name":"School of Computer Science, Zhuhai College of Science and Technology, Zhuhai 519041, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhuoxi","family":"Mai","sequence":"additional","affiliation":[{"name":"School of Computer Science, Zhuhai College of Science and Technology, Zhuhai 519041, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dalin","family":"Li","sequence":"additional","affiliation":[{"name":"School of Computer Science, Zhuhai College of Science and Technology, Zhuhai 519041, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guifang","family":"Fu","sequence":"additional","affiliation":[{"name":"Department of Applied Psychology, Guangdong University of Foreign Studies, Guangzhou 510006, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8403-6833","authenticated-orcid":false,"given":"Xueling","family":"Yang","sequence":"additional","affiliation":[{"name":"Department of Psychology, School of Public Health, Southern Medical University, Guangzhou 510515, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,6,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"561","DOI":"10.1037\/0022-006X.61.4.561","article-title":"The role of the therapeutic alliance in psychotherapy","volume":"61","author":"Horvath","year":"1993","journal-title":"J. Consult. Clin. Psychol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"252","DOI":"10.1037\/h0085885","article-title":"The generalizability of the psychoanalytic concept of the working alliance","volume":"16","author":"Bordin","year":"1979","journal-title":"Psychother. Theory Res. Pract."},{"key":"ref_3","unstructured":"Horvath, A.O. (1981). An Exploratory Study of the Working Alliance: Its Measurement and Relationship to Therapy Outcome. [Ph.D. Thesis, University of British Columbia]."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"438","DOI":"10.1037\/cou0000382","article-title":"Machine learning and natural language processing in psychotherapy research: Alliance as example use case","volume":"67","author":"Goldberg","year":"2020","journal-title":"J. Couns. Psychol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1016\/0306-4573(88)90021-0","article-title":"Term-weighting approaches in automatic text retrieval","volume":"24","author":"Salton","year":"1988","journal-title":"Inf. Process. Manag."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1100","DOI":"10.1080\/10503307.2022.2078680","article-title":"Predicting first session working alliances using deep learning algorithms: A proof-of-concept study for personalized psychotherapy","volume":"32","author":"Zhou","year":"2022","journal-title":"Psychother. Res."},{"key":"ref_7","unstructured":"Xie, H., Chen, Y., Xing, X., Lin, J., and Xu, X. (August, January 27). PsyDT: Using LLMs to construct the digital twin of psychological counselor with personalized counseling style for psychological counseling. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, Vienna, Austria."},{"key":"ref_8","unstructured":"Chaszczewicz, A., Shah, R.S., Louie, R., Arnow, B.A., Kraut, R., and Yang, D. (2024, January 11\u201316). Multi-level feedback generation with large language models for empowering novice peer counselors. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, Bangkok, Thailand."},{"key":"ref_9","unstructured":"Chen, K., Sun, Z., Wen, Y., Lian, H., Gao, Y., and Li, Y. (2025). Psy-Insight: Explainable multi-turn bilingual dataset for mental health counseling. arXiv."},{"key":"ref_10","unstructured":"Zhang, C., Li, R., Tan, M., Yang, M., Zhu, J., Yang, D., Zhao, J., Ye, G., Li, C., and Hu, X. (2024, January 11\u201316). CPsyCoun: A report-based multi-turn dialogue reconstruction and evaluation framework for Chinese psychological counseling. Proceedings of the Findings of the Association for Computational Linguistics: ACL 2024, Bangkok, Thailand."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"633","DOI":"10.1038\/s41586-025-09422-z","article-title":"DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning","volume":"645","author":"Guo","year":"2025","journal-title":"Nature"},{"key":"ref_12","unstructured":"Luo, R., Xu, J., Zhang, Y., Zhang, Z., Ren, X., and Sun, X. (2019). PKUSEG: A toolkit for multi-domain Chinese word segmentation. arXiv."},{"key":"ref_13","unstructured":"Honnibal, M., Montani, I., Van Landeghem, S., and Boyd, A. (2025, March 03). spaCy: Industrial-Strength Natural Language Processing in Python. Available online: https:\/\/github.com\/explosion\/spaCy."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1037\/cou0000433","article-title":"Language style matching in psychotherapy: An implicit aspect of alliance","volume":"67","author":"Doorn","year":"2020","journal-title":"J. Couns. Psychol."},{"key":"ref_15","first-page":"2825","article-title":"Scikit-learn: Machine learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"066138","DOI":"10.1103\/PhysRevE.69.066138","article-title":"Estimating mutual information","volume":"69","author":"Kraskov","year":"2004","journal-title":"Phys. Rev. E"},{"key":"ref_18","unstructured":"Chen, J., Xiao, S., Zhang, P., Luo, K., Lian, D., and Liu, Z. (2024, January 11\u201316). M3-Embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. Proceedings of the Findings of the Association for Computational Linguistics: ACL 2024, Bangkok, Thailand."},{"key":"ref_19","unstructured":"Chen, T., and Guestrin, C. (2016, January 13\u201317). XGBoost: A scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"286","DOI":"10.1002\/j.1556-6678.2002.tb00193.x","article-title":"Learning to write case notes using the SOAP format","volume":"80","author":"Cameron","year":"2002","journal-title":"J. Couns. Dev."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Reiter, M.D. (2023). A Therapist\u2019s Guide to Writing in Psychotherapy: Assessment, Documentation, and Intervention, Routledge. [1st ed.].","DOI":"10.4324\/9781003294702"},{"key":"ref_22","unstructured":"Norcross, J.C., and Goldfried, M.R. (2019). Handbook of Psychotherapy Integration, Oxford University Press. [3rd ed.]."},{"key":"ref_23","unstructured":"Akiba, T., Sano, S., Yanase, T., Ohta, T., and Koyama, M. (2019, January 4\u20138). Optuna: A next-generation hyperparameter optimization framework. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Anchorage, AK, USA."},{"key":"ref_24","unstructured":"Martinez, V.R., Flemotomos, N., Ardulov, V., Somandepalli, K., Goldberg, S.B., Imel, Z.E., Atkins, D.C., and Narayanan, S. (2019, January 15\u201319). Identifying therapist and client personae for therapeutic alliance estimation. Proceedings of the Interspeech 2019, Graz, Austria."},{"key":"ref_25","unstructured":"OpenAI (2024). GPT-4o system card. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"256","DOI":"10.1080\/10503307.2024.2428702","article-title":"Predicting working alliance in psychotherapy: A multi-modal machine learning approach","volume":"35","author":"Cicconet","year":"2025","journal-title":"Psychother. Res."},{"key":"ref_27","unstructured":"Qiu, H., and Lan, Z. (August, January 27). PsyDial: A large-scale long-term conversational dataset for mental health support. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, Vienna, Austria."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/28\/6\/699\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,23]],"date-time":"2026-06-23T04:29:57Z","timestamp":1782188997000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/28\/6\/699"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,17]]},"references-count":27,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2026,6]]}},"alternative-id":["e28060699"],"URL":"https:\/\/doi.org\/10.3390\/e28060699","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,17]]}}}