{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T12:18:37Z","timestamp":1779365917622,"version":"3.53.0"},"reference-count":59,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2025,5,3]],"date-time":"2025-05-03T00:00:00Z","timestamp":1746230400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,5,3]],"date-time":"2025-05-03T00:00:00Z","timestamp":1746230400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100008530","name":"European Regional Development Fund","doi-asserted-by":"publisher","award":["ED431G\/01"],"award-info":[{"award-number":["ED431G\/01"]}],"id":[{"id":"10.13039\/501100008530","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100008530","name":"European Regional Development Fund","doi-asserted-by":"publisher","award":["ED431G\/01"],"award-info":[{"award-number":["ED431G\/01"]}],"id":[{"id":"10.13039\/501100008530","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004837","name":"Ministerio de Ciencia e Innovaci\u00f3n","doi-asserted-by":"publisher","award":["PID2022-137061OB-C21"],"award-info":[{"award-number":["PID2022-137061OB-C21"]}],"id":[{"id":"10.13039\/501100004837","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004837","name":"Ministerio de Ciencia e Innovaci\u00f3n","doi-asserted-by":"publisher","award":["PLEC2021-007662"],"award-info":[{"award-number":["PLEC2021-007662"]}],"id":[{"id":"10.13039\/501100004837","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004837","name":"Ministerio de Ciencia e Innovaci\u00f3n","doi-asserted-by":"publisher","award":["PID2022-137061OB-C21"],"award-info":[{"award-number":["PID2022-137061OB-C21"]}],"id":[{"id":"10.13039\/501100004837","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004837","name":"Ministerio de Ciencia e Innovaci\u00f3n","doi-asserted-by":"publisher","award":["PLEC2021-007662"],"award-info":[{"award-number":["PLEC2021-007662"]}],"id":[{"id":"10.13039\/501100004837","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100014597","name":"Universidade da Coru\u00f1a","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100014597","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Lang Resources &amp; Evaluation"],"published-print":{"date-parts":[[2025,9]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Computational methods for depression detection aim to mine traces of depression from online publications posted by Internet users. However, solutions trained on existing collections exhibit limited generalisation and interpretability. To tackle these issues, recent studies have shown that identifying specific depressive symptoms can lead to more robust and effective models. The eRisk initiative fosters research on this area and has recently proposed a new ranking task focused on developing search methods to find sentences related to depressive symptoms. This search challenge relies on the symptoms specified by the Beck Depression Inventory-II (BDI-II), a questionnaire widely used in clinical practice. It includes symptoms such as sadness, irritability or lack of sleep. Given the input submitted by systems participating in eRisk, we first apply top-k pooling over the systems\u2019 relevance rankings, obtaining a diverse set of sentences. These sentences are judged for relevance, leading to <jats:italic>DepreSym<\/jats:italic>, a dataset consisting of 21,580 sentences annotated according to their relevance to the 21 BDI-II symptoms. This dataset serves as a valuable resource for advancing the development of models that monitor depression markers. Due to the complex nature of this relevance annotation, we designed a robust assessment methodology carried out by three expert assessors, including a trained psychologist. As part of this study, we explore the potential of recent Large Language Models (ChatGPT, GPT4 and Vicuna) as assessors in this complex task. We undertake a comprehensive examination of the LLMs\u2019 performance, studying their main limitations and analysing their role as a complement or replacement for human annotators. Finally, we incorporate our dataset into the Benchmarking Information Retrieval (BEIR) framework for a thorough search evaluation. We use state-of-the-art retrieval systems, including lexical, sparse, dense and re-ranking architectures, to gain insights about the dataset\u2019s complexity and identify potential avenues for improvement.<\/jats:p>","DOI":"10.1007\/s10579-025-09831-6","type":"journal-article","created":{"date-parts":[[2025,5,3]],"date-time":"2025-05-03T01:52:34Z","timestamp":1746237154000},"page":"2737-2762","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["DepreSym: A Depression Symptom Annotated Corpus and the Role of Large Language Models as Assessors of Psychological Markers"],"prefix":"10.1007","volume":"59","author":[{"given":"Anxo","family":"P\u00e9rez","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marcos","family":"Fern\u00e1ndez-Pichel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Javier","family":"Parapar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David E.","family":"Losada","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,5,3]]},"reference":[{"key":"9831_CR1","unstructured":"Alonso, O., & Mizzaro, S. (2009). Can we get rid of TREC assessors? Using mechanical Turk for relevance assessment. In Proceedings of the SIGIR 2009 workshop on the future of IR evaluation (Vol. 15, p. 16). Boston, Massachusetts, USA."},{"key":"9831_CR2","doi-asserted-by":"crossref","unstructured":"Babakov, N., Logacheva, V., & Panchenko, A. (2023). Beyond plain toxic: Building datasets for detection of flammable topics and inappropriate statements. Language Resources and Evaluation, 1\u201346.","DOI":"10.1007\/s10579-023-09682-z"},{"issue":"3","key":"9831_CR3","doi-asserted-by":"publisher","first-page":"588","DOI":"10.1207\/s15327752jpa6703_13","volume":"67","author":"AT Beck","year":"1996","unstructured":"Beck, A. T., Steer, R. A., Ball, R., & Ranieri, W. F. (1996). Comparison of beck depression inventories-IA and-II in psychiatric outpatients. Journal of Personality Assessment, 67(3), 588\u2013597.","journal-title":"Journal of Personality Assessment"},{"key":"9831_CR4","doi-asserted-by":"crossref","unstructured":"Berger, A., Caruana, R., Cohn, D., Freitag, D., & Mittal, V. (2020). Bridging the lexical chasm, statistical approaches to answer-finding. In Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval (pp. 192\u2013199). Athens, Greece.","DOI":"10.1145\/345508.345576"},{"key":"9831_CR5","doi-asserted-by":"crossref","unstructured":"B\u00fcttcher, S., Clarke, C. L., Yeung, P. C., & Soboroff, I. (2007). Reliable information retrieval evaluation with incomplete and biased judgements. In Proceedings of the 30th annual international ACM SIGIR conference on research and development in information retrieval (pp. 63\u201370). Amsterdam, Netherlands,","DOI":"10.1145\/1277741.1277755"},{"issue":"1","key":"9831_CR6","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1038\/s41746-020-0233-7","volume":"3","author":"S Chancellor","year":"2020","unstructured":"Chancellor, S., & De Choudhury, M. (2020). Methods in predictive techniques for mental health status on social media: A critical review. NPJ Digital Medicine, 3(1), 43.","journal-title":"NPJ Digital Medicine"},{"key":"9831_CR7","unstructured":"Chiang, W.-L., Li, Z., Lin, Z., Sheng, Y., Wu, Z., Zhang, H., Zheng, L., Zhuang, S., Zhuang, Y., Gonzalez, J. E., Stoica, I., & Xing, E. P. (2023). Vicuna: An open-source chatbot impressing GPT-4 with 90%* chatgpt quality."},{"key":"9831_CR8","doi-asserted-by":"publisher","first-page":"117822261879286","DOI":"10.1177\/1178222618792860","volume":"10","author":"G Coppersmith","year":"2018","unstructured":"Coppersmith, G., Leary, R., Crutchley, P., & Fine, A. (2018). Natural language processing of social media as screening for suicide risk. Biomedical Informatics Insights, 10, 1178222618792860.","journal-title":"Biomedical Informatics Insights"},{"key":"9831_CR9","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-031-04431-1","volume-title":"Early detection of mental health disorders by social media monitoring: The first five years of the ERisk project","author":"F Crestani","year":"2022","unstructured":"Crestani, F., Losada, D. E., & Parapar, J. (2022). Early detection of mental health disorders by social media monitoring: The first five years of the ERisk project (Vol. 1018). Springer."},{"key":"9831_CR10","doi-asserted-by":"crossref","unstructured":"Dai, Z., & Callan, J. (2020). Context-aware term weighting for first stage passage retrieval. In Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval (pp. 1533\u20131536). Online Event.","DOI":"10.1145\/3397271.3401204"},{"key":"9831_CR11","unstructured":"Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805."},{"key":"9831_CR12","doi-asserted-by":"crossref","unstructured":"Faggioli, G., Dietz, L., Clarke, C. L., Demartini, G., Hagen, M., Hauff, C., Kando, N., Kanoulas, E., Potthast, M., Stein, B., & Wachsmuth, H. (2023). Perspectives on large language models for relevance judgment. arXiv preprint arXiv:2304.09161.","DOI":"10.1145\/3578337.3605136"},{"key":"9831_CR13","unstructured":"Forbes. (2022). Introducing chatgpt, November https:\/\/openai.com\/blog\/chatgpt. [Accessed April 4, 2023]."},{"key":"9831_CR14","doi-asserted-by":"crossref","unstructured":"Gilardi, F., Alizadeh, M., & Kubli, M. (2023). ChatGPT outperforms crowd-workers for text-annotation tasks. arXiv preprint arXiv:2303.15056.","DOI":"10.1073\/pnas.2305016120"},{"key":"9831_CR15","doi-asserted-by":"crossref","unstructured":"Harrigian, K., Aguirre, C., & Dredze, M. (2020). Do models of mental health based on social media data generalize? In Findings of the association for computational linguistics: EMNLP 2020 (pp. 3774\u20133788). Online Event.","DOI":"10.18653\/v1\/2020.findings-emnlp.337"},{"key":"9831_CR16","doi-asserted-by":"crossref","unstructured":"He, X., Lin, Z., Gong, Y., Jin, A., Zhang, H., Lin, C., Jiao, J., Yiu, S. M., Duan, N., & Chen, W. (2023). Annollm: Making large language models to be better crowdsourced annotators. arXiv preprint arXiv:2303.16854.","DOI":"10.18653\/v1\/2024.naacl-industry.15"},{"key":"9831_CR17","doi-asserted-by":"crossref","unstructured":"Hofst\u00e4tter, Lin, S. C., Yang, J. H., Lin, J., & Hanbury, A. (2021). Efficiently teaching an effective dense retriever with balanced topic aware sampling. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval (pp. 113\u2013122). Online Event.","DOI":"10.1145\/3404835.3462891"},{"key":"9831_CR18","doi-asserted-by":"crossref","unstructured":"Karpukhin, V., Oguz, B., Min, S., Lewis, P. S., Wu, L., Edunov, S., Chen, D., & Yih, W. T. (2020). Dense passage retrieval for open-domain question answering. arXiv preprint arXiv:2004.04906.","DOI":"10.18653\/v1\/2020.emnlp-main.550"},{"key":"9831_CR19","doi-asserted-by":"crossref","unstructured":"Khattab, O., & Zaharia, M. (2020). Colbert: Efficient and effective passage search via contextualized late interaction over Bert. In Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval (pp. 39\u201348). Xi\u2019an, China.","DOI":"10.1145\/3397271.3401075"},{"key":"9831_CR20","volume-title":"Content analysis: An introduction to its methodology","author":"K Krippendorff","year":"2018","unstructured":"Krippendorff, K. (2018). Content analysis: An introduction to its methodology. Sage publications."},{"key":"9831_CR21","doi-asserted-by":"crossref","unstructured":"Losada, D. E., Crestani, F., & Parapar, J. (2018). Overview of erisk 2018: Early risk prediction on the internet (extended lab overview). In Proceedings of the 9th international conference of the CLEF association, CLEF 2018, Avignon, France (pp. 1\u201320).","DOI":"10.1007\/978-3-319-98932-7_30"},{"key":"9831_CR22","doi-asserted-by":"crossref","unstructured":"Losada, D. E., Crestani, F., & Parapar, J. (2019). Overview of erisk 2019 early risk prediction on the internet. In Experimental IR meets multilinguality, multimodality, and interaction: 10th international conference of the CLEF Association, CLEF 2019, Lugano, Switzerland, September 9\u201312, 2019, proceedings 10 (pp. 340\u2013357). Springer.","DOI":"10.1007\/978-3-030-28577-7_27"},{"key":"9831_CR23","doi-asserted-by":"crossref","unstructured":"Losada, D. E., Crestani, F., & Parapar, J. (2020). Erisk 2020: Self-harm and depression challenges. In 42nd European conference on information retrieval, ECIR 2020, Lisbon, Portugal (pp. 557\u2013563). Springer.","DOI":"10.1007\/978-3-030-45442-5_72"},{"key":"9831_CR24","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10579-018-9423-1","volume":"54","author":"DE Losada","year":"2020","unstructured":"Losada, D. E., & Gamallo, P. (2020). Evaluating and improving lexical resources for detecting signs of depression in text. Language Resources and Evaluation, 54, 1\u201324.","journal-title":"Language Resources and Evaluation"},{"key":"9831_CR25","doi-asserted-by":"crossref","unstructured":"MacAvaney, S., Desmet, B., Cohan, A., Soldaini, L., Yates, A., Zirikly, A., & Goharian, N. (2018). RSDD-time: Temporal annotation of self-reported mental health diagnoses. In Proceedings of the fifth workshop on computational linguistics and clinical psychology: From keyboard to clinic, New Orleans, LA (pp. 168\u2013173). Association for Computational Linguistics.","DOI":"10.18653\/v1\/W18-0618"},{"issue":"3","key":"9831_CR26","doi-asserted-by":"publisher","first-page":"276","DOI":"10.11613\/BM.2012.031","volume":"22","author":"ML McHugh","year":"2012","unstructured":"McHugh, M. L. (2012). Interrater reliability: The kappa statistic. Biochemia Medica, 22(3), 276\u2013282.","journal-title":"Biochemia Medica"},{"key":"9831_CR27","doi-asserted-by":"crossref","unstructured":"Meyer, S., Elsweiler, D., Ludwig, B., Fernandez-Pichel, M., & Losada, D. E. (2022). Do we still need human assessors? Prompt-based Gpt-3 user simulation in conversational AI. In Proceedings of the 4th conference on conversational user interfaces, Glasgow, Scotland (pp. 1\u20136).","DOI":"10.1145\/3543829.3544529"},{"issue":"2","key":"9831_CR28","doi-asserted-by":"publisher","first-page":"301","DOI":"10.1016\/j.joi.2012.12.001","volume":"7","author":"SI Moghadasi","year":"2013","unstructured":"Moghadasi, S. I., Ravana, S. D., & Raman, S. N. (2013). Low-cost evaluation techniques for information retrieval systems: A review. Journal of Informetrics, 7(2), 301\u2013312.","journal-title":"Journal of Informetrics"},{"key":"9831_CR29","doi-asserted-by":"crossref","unstructured":"Neuman, Y., Cohen, Y., Assaf, D., & Kedma, G. (2012). Proactive screening for depression through metaphorical and automatic text analysis. Artificial Intelligence in Medicine,56(1), 19\u201325. ISSN 0933-3657.","DOI":"10.1016\/j.artmed.2012.06.001"},{"key":"9831_CR30","doi-asserted-by":"crossref","unstructured":"Nguyen, T., Yates, A., Zirikly, A., Desmet, B., & Cohan, A. (2022a). Improving the generalizability of depression detection by leveraging clinical questionnaires. In S. Muresan, P. Nakov, & A. Villavicencio (Eds.), Proceedings of the 60th annual meeting of the association for computational linguistics, ACL 2022, Dublin, Ireland (pp. 8446\u20138459). Association for Computational Linguistics.","DOI":"10.18653\/v1\/2022.acl-long.578"},{"key":"9831_CR31","doi-asserted-by":"crossref","unstructured":"Nguyen, T., Yates, A., Zirikly, A., Desmet, B., & Cohan, A. (2022b). Improving the generalizability of depression detection by leveraging clinical questionnaires. In Proceedings of the 60th annual meeting of the association for computational linguistics, Dublin, Ireland (pp. 8446\u20138459). Association for Computational Linguistics.","DOI":"10.18653\/v1\/2022.acl-long.578"},{"key":"9831_CR32","unstructured":"Nogueira, R., Lin, J., & Epistemic, A. I. (2019). From doc2query to doctttttquery. Online Preprint,6, 2."},{"key":"9831_CR33","volume-title":"The diagnostic and statistical manual of mental disorders, (DSM-5)","author":"CC Nuckols","year":"2013","unstructured":"Nuckols, C. C., & Nuckols, C. C. (2013). The diagnostic and statistical manual of mental disorders, (DSM-5). American Psychiatric Association."},{"key":"9831_CR34","unstructured":"OpenAI. (2023). GPT-4 technical report. arXiv:submit\/4812508."},{"key":"9831_CR35","doi-asserted-by":"crossref","unstructured":"Parapar, J., Mart\u00edn-Rodilla, P., Losada, D. E., & Crestani, F. (2021). Overview of eRisk 2021: Early risk prediction on the internet. In International conference of the cross-language evaluation forum for European languages (pp. 324\u2013344). Springer.","DOI":"10.1007\/978-3-030-85251-1_22"},{"key":"9831_CR36","doi-asserted-by":"crossref","unstructured":"Parapar, J., Mart\u00edn-Rodilla, P., Losada, D. E., & Crestani, F. (2023). Overview of erisk 2023: Early risk prediction on the internet. In International conference of the cross-language evaluation forum for European languages, Thessaloniki, Greece (pp. 294\u2013315). Springer.","DOI":"10.1007\/978-3-031-42448-9_22"},{"key":"9831_CR37","doi-asserted-by":"publisher","DOI":"10.1016\/S0262-4079(11)62167-2","volume-title":"The secret life of pronouns: What our words say about us","author":"JW Pennebaker","year":"2011","unstructured":"Pennebaker, J. W. (2011). The secret life of pronouns: What our words say about us. Bloomsbury Press."},{"key":"9831_CR38","doi-asserted-by":"crossref","unstructured":"P\u00e9rez, A., Parapar, J., & Barreiro, \u00c1. (2022). Automatic depression score estimation with word embedding models. Artificial Intelligence in Medicine, 132, 102380. ISSN 0933-3657.","DOI":"10.1016\/j.artmed.2022.102380"},{"key":"9831_CR39","doi-asserted-by":"crossref","unstructured":"P\u00e9rez, A., Parapar, J., Barreiro, \u00c1., & Lopez-Larrosa, S. (2023) Bdi-sen: A sentence dataset for clinical symptoms of depression. In Proceedings of the 46th international ACM SIGIR conference on research and development in information retrieval, Taipei, Taiwan (pp. 2996\u20133006). Association for Computing Machinery. ISBN 9781450394086.","DOI":"10.1145\/3539618.3591905"},{"key":"9831_CR40","doi-asserted-by":"crossref","unstructured":"P\u00e9rez, A., Warikoo, N., Wang, K., Parapar, J., & Gurevych, I. (2023). Semantic similarity models for depression severity estimation. arXiv preprint arXiv:2211.07624 To appear in: EMNLP.","DOI":"10.18653\/v1\/2023.emnlp-main.1000"},{"key":"9831_CR41","doi-asserted-by":"crossref","unstructured":"Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 conference on empirical methods in natural language processing, EMNLP 2019, Hong Kong. Association for Computational Linguistics.","DOI":"10.18653\/v1\/D19-1410"},{"key":"9831_CR42","doi-asserted-by":"crossref","unstructured":"R\u00edssola, E. A., Losada, D. E., & Crestani, F. (2021). A survey of computational methods for online mental state assessment on social media. ACM Transactions on Computing for Healthcare, 2(2). ISSN 2691-1957.","DOI":"10.1145\/3437259"},{"key":"9831_CR43","doi-asserted-by":"crossref","unstructured":"Robertson, S. E., Walker, S., Jones, S., Hancock-Beaulieu, M. M., & Gatford, M. (1995). Okapi at TREC-3. NIST Special Publication Sp, 109, 109.","DOI":"10.6028\/NIST.SP.500-225.city"},{"key":"9831_CR44","doi-asserted-by":"publisher","first-page":"156","DOI":"10.2197\/ipsjjip.17.156","volume":"17","author":"T Sakai","year":"2009","unstructured":"Sakai, T. (2009). On the robustness of information retrieval metrics to biased relevance assessments. Journal of Information Processing, 17, 156\u2013166.","journal-title":"Journal of Information Processing"},{"key":"9831_CR45","doi-asserted-by":"crossref","unstructured":"Santos, W. R. D., de\u00a0Oliveira, R. L., & Paraboni, I. (2023). Setembrobr: A social media corpus for depression and anxiety disorder prediction. Language Resources and Evaluation, 1\u201328.","DOI":"10.1007\/s10579-022-09633-0"},{"key":"9831_CR46","doi-asserted-by":"crossref","unstructured":"Scholer, F., Turpin, A., & Sanderson, M. (2011). Quantifying test collection quality based on the consistency of relevance judgements. In Proceedings of the 34th international ACM SIGIR conference on research and development in information retrieval, Beijing, China (pp. 1063\u20131072).","DOI":"10.1145\/2009916.2010057"},{"key":"9831_CR47","doi-asserted-by":"crossref","unstructured":"Steer, R. A., Beck, A. T., & Garrison, B. (1986). Applications of the beck depression inventory. In Assessment of depression (pp. 123\u2013142). Springer.","DOI":"10.1007\/978-3-642-70486-4_13"},{"key":"9831_CR48","unstructured":"Thakur, N., Reimers, N., R\u00fcckl\u00e9, A., Srivastava, A., & Gurevych, I. (2021). BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Thirty-fifth conference on neural information processing systems datasets and benchmarks track, NeurIPS 2021. Online Event."},{"key":"9831_CR49","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, \u0141., & Polosukhin, I. (2017). Attention is all you need. In Advances in neural information processing systems, NeurIPS 2017 (Vol. 30)."},{"issue":"1","key":"9831_CR50","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1093\/jamiaopen\/ooz054","volume":"3","author":"CG Walsh","year":"2020","unstructured":"Walsh, C. G., Chaudhry, B., Dua, P., Goodman, K. W., Kaplan, B., Kavuluru, R., Solomonides, A., & Subbian, V. (2020). Stigma, biomarkers, and algorithmic bias: Recommendations for precision behavioral health with artificial intelligence. JAMIA Open, 3(1), 9\u201315.","journal-title":"JAMIA Open"},{"key":"9831_CR51","unstructured":"Wang, W., Wei, F., Li, D., Bao, H., Yang, N., & Zhou, M. (2020). Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. In Advances in neural information processing systems, NeurIPS, 2020 (Vol. 33, pp. 5776\u20135788)."},{"key":"9831_CR52","unstructured":"Wolohan, J. T., Hiraga, M., Mukherjee, A., Sayyed, Z. A., & Millard, M. (2018). Detecting linguistic traces of depression in topic-restricted text: Attending to self-stigmatized depression with NLP. In Proceedings of the first international workshop on language cognition and computational models, New Mexico, USA (pp. 11\u201321)."},{"key":"9831_CR53","unstructured":"Xiong, L., Xiong, C., Li, Y., Tang, K. F., Liu, J., Bennett, P., Ahmed, J., & Overwijk, A. (2020). Approximate nearest neighbor negative contrastive learning for dense text retrieval. arXiv preprint arXiv:2007.00808."},{"key":"9831_CR54","doi-asserted-by":"crossref","unstructured":"Yang, P., Fang, H., & Lin, J. (2017). Anserini: Enabling the use of lucene for information retrieval research. In Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval, Tokyo, Japan (pp. 1253\u20131256).","DOI":"10.1145\/3077136.3080721"},{"key":"9831_CR55","doi-asserted-by":"crossref","unstructured":"Yilmaz, E., Aslam, J. A., & Robertson, S. (2008). A new rank correlation coefficient for information retrieval. In S.-H. Myaeng, D. W. Oard, F. Sebastiani, T.-S. Chua, & M.-K. Leong (Eds.), Proceedings of the 31st annual international ACM SIGIR conference on research and development in information retrieval, SIGIR 2008, Singapore, Singapore (pp. 587\u2013594). ACM.","DOI":"10.1145\/1390334.1390435"},{"key":"9831_CR56","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Chen, S., Wu, M., & Zhu, K. (2022a). Symptom identification for interpretable detection of multiple mental disorders on social media. In Proceedings of the 2022 conference on empirical methods in natural language processing, Abu Dhabi, United Arab Emirates (pp. 9970\u20139985). Association for Computational Linguistics.","DOI":"10.18653\/v1\/2022.emnlp-main.677"},{"key":"9831_CR57","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Chen, S., Wu, M., & Zhu, K. Q. (2022b). Symptom identification for interpretable detection of multiple mental disorders on social media. In Y. Goldberg, Z. Kozareva, & Y. Zhang (Eds.), Proceedings of the 2022 conference on empirical methods in natural language processing, EMNLP 2022, Abu Dhabi, United Arab Emirates (pp. 9970\u20139985). Association for Computational Linguistics.","DOI":"10.18653\/v1\/2022.emnlp-main.677"},{"key":"9831_CR58","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Chen, S., Wu, M., & Zhu, K. Q. (2022c). Psychiatric scale guided risky post screening for early detection of depression. In L. De Raedt (Ed.), Proceedings of the thirty-first international joint conference on artificial intelligence, IJCAI 2022, Vienna, Austria (pp. 5220\u20135226).","DOI":"10.24963\/ijcai.2022\/725"},{"key":"9831_CR59","doi-asserted-by":"crossref","unstructured":"Zhao, T., Lu, X., & Lee, K. (2020). Sparta: Efficient open-domain question answering via sparse transformer matching retrieval. arXiv preprint arXiv:2009.13013.","DOI":"10.18653\/v1\/2021.naacl-main.47"}],"container-title":["Language Resources and Evaluation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10579-025-09831-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10579-025-09831-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10579-025-09831-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,6]],"date-time":"2025-09-06T12:41:33Z","timestamp":1757162493000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10579-025-09831-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,3]]},"references-count":59,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2025,9]]}},"alternative-id":["9831"],"URL":"https:\/\/doi.org\/10.1007\/s10579-025-09831-6","relation":{},"ISSN":["1574-020X","1574-0218"],"issn-type":[{"value":"1574-020X","type":"print"},{"value":"1574-0218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,3]]},"assertion":[{"value":"16 April 2025","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 May 2025","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"We declare that we have no conflict of interest as defined by Springer or other interests that might be perceived to influence the results and\/or discussion reported in this paper.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}