{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T16:33:18Z","timestamp":1781281998988,"version":"3.54.1"},"reference-count":86,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2022,3,7]],"date-time":"2022-03-07T00:00:00Z","timestamp":1646611200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,3,7]],"date-time":"2022-03-07T00:00:00Z","timestamp":1646611200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000781","name":"European Research Council","doi-asserted-by":"publisher","award":["957356"],"award-info":[{"award-number":["957356"]}],"id":[{"id":"10.13039\/501100000781","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100011736","name":"Ministerium f\u00fcr Wirtschaft, Arbeit und Wohnungsbau Baden-W\u00fcrttemberg","doi-asserted-by":"publisher","award":["MultiConvAI"],"award-info":[{"award-number":["MultiConvAI"]}],"id":[{"id":"10.13039\/501100011736","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100007105","name":"Universit\u00e4t Mannheim","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100007105","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Inf Retrieval J"],"published-print":{"date-parts":[[2022,6]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Pretrained multilingual text encoders based on neural <jats:italic>transformer architectures<\/jats:italic>, such as multilingual BERT (mBERT) and XLM, have recently become a default paradigm for cross-lingual transfer of natural language processing models, rendering cross-lingual word embedding spaces (CLWEs) effectively obsolete. In this work we present a systematic empirical study focused on the suitability of the state-of-the-art multilingual encoders for cross-lingual document and sentence retrieval tasks across a number of diverse language pairs. We first treat these models as multilingual text encoders and benchmark their performance in unsupervised ad-hoc sentence- and document-level CLIR. In contrast to supervised language understanding, our results indicate that for unsupervised document-level CLIR\u2014a setup with no relevance judgments for IR-specific fine-tuning\u2014pretrained multilingual encoders on average fail to significantly outperform earlier models based on CLWEs. For sentence-level retrieval, we do obtain state-of-the-art performance: the peak scores, however, are met by multilingual encoders that have been further specialized, in a supervised fashion, for sentence understanding tasks, rather than using their vanilla \u2018off-the-shelf\u2019 variants. Following these results, we introduce localized relevance matching for document-level CLIR, where we independently score a query against document sections. In the second part, we evaluate multilingual encoders fine-tuned in a supervised fashion (i.e., we <jats:italic>learn to rank<\/jats:italic>) on English relevance data in a series of zero-shot language and domain transfer CLIR experiments. Our results show that, despite the supervision, and due to the domain and language shift, supervised re-ranking rarely improves the performance of multilingual transformers as unsupervised base rankers. Finally, only with in-domain contrastive fine-tuning (i.e., same domain, only language transfer), we manage to improve the ranking quality. We uncover substantial empirical differences between cross-lingual retrieval results and results of (zero-shot) cross-lingual transfer for monolingual retrieval in target languages, which point to \u201cmonolingual overfitting\u201d of retrieval models trained on monolingual (English) data, even if they are based on multilingual transformers.<\/jats:p>","DOI":"10.1007\/s10791-022-09406-x","type":"journal-article","created":{"date-parts":[[2022,3,7]],"date-time":"2022-03-07T08:04:33Z","timestamp":1646640273000},"page":"149-183","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":27,"title":["On cross-lingual retrieval with multilingual text encoders"],"prefix":"10.1007","volume":"25","author":[{"given":"Robert","family":"Litschko","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ivan","family":"Vuli\u0107","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Simone Paolo","family":"Ponzetto","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Goran","family":"Glava\u0161","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2022,3,7]]},"reference":[{"key":"9406_CR1","doi-asserted-by":"crossref","unstructured":"Akkalyoncu\u00a0Yilmaz, Z., Yang, W., Zhang, H., & Lin, J. (2019). Cross-domain modeling of sentence-level evidence for document retrieval. In Proceedings of EMNLP, pp. 3490\u20133496.","DOI":"10.18653\/v1\/D19-1352"},{"key":"9406_CR2","doi-asserted-by":"crossref","unstructured":"Artetxe, M., Labaka, G., & Agirre, E. (2018). A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings. In Proceedings of ACL, pp. 789\u2013798.","DOI":"10.18653\/v1\/P18-1073"},{"key":"9406_CR3","doi-asserted-by":"crossref","unstructured":"Artetxe, M., & Schwenk, H. (2019). Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond. Transactions of the ACL pp. 597\u2013610.","DOI":"10.1162\/tacl_a_00288"},{"key":"9406_CR4","unstructured":"Beltagy, I., Peters, M.E., & Cohan, A. (2020). Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150"},{"key":"9406_CR5","doi-asserted-by":"crossref","unstructured":"Braschler, M. (2003). CLEF 2003\u2013Overview of results. In Workshop of the cross-language evaluation forum for european languages, pp. 44\u201363.","DOI":"10.1007\/978-3-540-30222-3_5"},{"key":"9406_CR6","unstructured":"Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., & Amodei, D. (2020). Language models are few-shot learners. In Proceedings of NeurIPS, pp. 1877\u20131901."},{"key":"9406_CR7","unstructured":"Cao, S., Kitaev, N., & Klein, D. (2020). Multilingual alignment of contextual word representations. In Proceedings of ICLR."},{"key":"9406_CR8","doi-asserted-by":"crossref","unstructured":"Cer, D., Diab, M., Agirre, E., Lopez-Gazpio, I., & Specia, L. (2017). SemEval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation. In Proceedings of SemEval, pp. 1\u201314.","DOI":"10.18653\/v1\/S17-2001"},{"key":"9406_CR9","doi-asserted-by":"crossref","unstructured":"Cer, D., Yang, Y., Kong, S.y., Hua, N., Limtiaco, N., St.\u00a0John, R., Constant, N., Guajardo-Cespedes, M., Yuan, S., Tar, C., Strope, B., & Kurzweil, R. (2018). Universal sentence encoder for English. In Proceedings of EMNLP, pp. 169\u2013174.","DOI":"10.18653\/v1\/D18-2029"},{"key":"9406_CR10","doi-asserted-by":"crossref","unstructured":"Chidambaram, M., Yang, Y., Cer, D., Yuan, S., Sung, Y., Strope, B., & Kurzweil, R. (2019). Learning cross-lingual sentence representations via a multi-task dual-encoder model. In Proceedings of ACL: Workshop on representation learning for NLP, pp. 250\u2013259.","DOI":"10.18653\/v1\/W19-4330"},{"key":"9406_CR11","unstructured":"Clark, K., Luong, M., Le, Q.V., & Manning, C. D. (2020). ELECTRA: Pre-training text encoders as discriminators rather than generators. In Proceedings of ICLR."},{"key":"9406_CR12","doi-asserted-by":"crossref","unstructured":"Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzm\u00e1n, F., Grave, \u00c9., Ott, M., Zettlemoyer, L., & Stoyanov, V. (2020). Unsupervised cross-lingual representation learning at scale. In Proceedings of ACL, pp. 8440\u20138451.","DOI":"10.18653\/v1\/2020.acl-main.747"},{"key":"9406_CR13","doi-asserted-by":"crossref","unstructured":"Conneau, A., Khandelwal, K., Goyal, N., Chaudhary, V., Wenzek, G., Guzm\u00e1n, F., Grave, E., Ott, M., Zettlemoyer, L., & Stoyanov, V. (2020). Unsupervised cross-lingual representation learning at scale. In Proceedings of ACL, pp. 8440\u20138451.","DOI":"10.18653\/v1\/2020.acl-main.747"},{"key":"9406_CR14","unstructured":"Conneau, A., & Kiela, D. (2018). SentEval: An evaluation toolkit for universal sentence representations. In Proceedings of LREC, pp. 1699\u20131704."},{"key":"9406_CR15","doi-asserted-by":"crossref","unstructured":"Conneau, A., Kiela, D., Schwenk, H., Barrault, L., & Bordes, A. (2017). Supervised learning of universal sentence representations from natural language inference data. In Proceedings of EMNLP, pp. 670\u2013680.","DOI":"10.18653\/v1\/D17-1070"},{"key":"9406_CR16","unstructured":"Conneau, A., & Lample, G. (2019). Cross-lingual language model pretraining. In Proceedings of NeurIPS, pp. 7059\u20137069."},{"key":"9406_CR17","doi-asserted-by":"crossref","unstructured":"Craswell, N., Mitra, B., Yilmaz, E., Campos, D., & Lin, J. (2021). Ms marco: Benchmarking ranking models in the large-data regime. In Proceedings of SIGIR, pp. 1566\u20131576.","DOI":"10.1145\/3404835.3462804"},{"key":"9406_CR18","doi-asserted-by":"crossref","unstructured":"Dai, Z., & Callan, J. (2019). Deeper text understanding for IR with contextual neural language modeling. In Proceedings of SIGIR, pp. 985\u2013988","DOI":"10.1145\/3331184.3331303"},{"key":"9406_CR19","doi-asserted-by":"crossref","unstructured":"Dai, Z., Yang, Z., Yang, Y., Carbonell, J., Le, Q., & Salakhutdinov, R. (2019). Transformer-XL: Attentive language models beyond a fixed-length context. In Proceedings of ACL, pp. 2978\u20132988.","DOI":"10.18653\/v1\/P19-1285"},{"key":"9406_CR20","unstructured":"Devlin, J., Chang, M.W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL, pp. 4171\u20134186."},{"key":"9406_CR21","doi-asserted-by":"crossref","unstructured":"Ethayarajh, K. (2019). How contextual are contextualized word representations? Comparing the geometry of BERT, ELMo, and GPT-2 embeddings. In Proceedings of EMNLP, pp. 55\u201365.","DOI":"10.18653\/v1\/D19-1006"},{"key":"9406_CR22","unstructured":"Feng, F., Yang, Y., Cer, D., Arivazhagan, N., & Wang, W. (2020). Language-agnostic BERT sentence embedding. arXiv preprint arXiv:2007.01852"},{"key":"9406_CR23","doi-asserted-by":"crossref","unstructured":"Gao, L., Dai, Z., & Callan, J. (2020). Modularized transfomer-based ranking framework. In Proceedings of EMNLP, pp. 4180\u20134190.","DOI":"10.18653\/v1\/2020.emnlp-main.342"},{"key":"9406_CR24","doi-asserted-by":"crossref","unstructured":"Glava\u0161, G., Litschko, R., Ruder, S., & Vuli\u0107, I. (2019). How to (properly) evaluate cross-lingual word embeddings: On strong baselines, comparative analyses, and some misconceptions. In Proceedings of ACL, pp. 710\u2013721.","DOI":"10.18653\/v1\/P19-1070"},{"key":"9406_CR25","doi-asserted-by":"crossref","unstructured":"Guo, M., Shen, Q., Yang, Y., Ge, H., Cer, D., Hernandez\u00a0Abrego, G., Stevens, K., Constant, N., Sung, Y.H., Strope, B., & Kurzweil, R. (2018). Effective parallel corpus mining using bilingual sentence embeddings. In Proceedings of WMT, pp. 165\u2013176.","DOI":"10.18653\/v1\/W18-6317"},{"key":"9406_CR26","doi-asserted-by":"crossref","unstructured":"Hoogeveen, D., Verspoor, K.M., & Baldwin, T. (2015). CQADupStack: A benchmark data set for community question-answering research. In Proceedings of ADCS, pp. 3:1\u20133:8.","DOI":"10.1145\/2838931.2838934"},{"key":"9406_CR27","unstructured":"Hu, J., Ruder, S., Siddhant, A., Neubig, G., Firat, O., & Johnson, M. (2020). Xtreme: A massively multilingual multi-task benchmark for evaluating cross-lingual generalisation. In Proceedings of ICML, pp. 4411\u20134421."},{"key":"9406_CR28","unstructured":"Humeau, S., Shuster, K., Lachaux, M.A., & Weston, J. (2020). Poly-encoders: Architectures and pre-training strategies for fast and accurate multi-sentence scoring. In Proceedings of ICLR."},{"key":"9406_CR29","unstructured":"Jiang, Z., El-Jaroudi, A., Hartmann, W., Karakos, D., & Zhao, L. (2020). Cross-lingual information retrieval with BERT. In Proceedings of LREC, p.\u00a026."},{"key":"9406_CR30","unstructured":"Johnson, J., Douze, M., & J\u00e9gou, H. (2017). Billion-scale similarity search with gpus. arXiv preprint arXiv:1702.08734"},{"key":"9406_CR31","unstructured":"Karthikeyan, K., Wang, Z., Mayhew, S., & Roth, D. (2020). Cross-lingual ability of multilingual BERT: An empirical study. In Proceedings of ICLR."},{"key":"9406_CR32","doi-asserted-by":"crossref","unstructured":"Khattab, O., & Zaharia, M. (2020). Colbert: Efficient and effective passage search via contextualized late interaction over bert. In Proceedings of SIGIR, pp. 39\u201348.","DOI":"10.1145\/3397271.3401075"},{"key":"9406_CR33","unstructured":"Koehn, P. (2005). Europarl: A parallel corpus for statistical machine translation. In Proceedings of the 10th machine translation summit (MT SUMMIT), pp. 79\u201386."},{"key":"9406_CR34","doi-asserted-by":"crossref","unstructured":"Lauscher, A., Ravishankar, V., Vuli\u0107, I., & Glava\u0161, G. (2020). From zero to hero: On the limitations of zero-shot language transfer with multilingual transformers. In Proceedings of EMNLP, pp. 4483\u20134499.","DOI":"10.18653\/v1\/2020.emnlp-main.363"},{"key":"9406_CR35","doi-asserted-by":"crossref","unstructured":"Lei, T., Joshi, H., Barzilay, R., Jaakkola, T., Tymoshenko, K., Moschitti, A., & M\u00e0rquez, L. (2016). Semi-supervised question retrieval with gated convolutions. In Proceedings of NAACL, pp. 1279\u20131289.","DOI":"10.18653\/v1\/N16-1153"},{"key":"9406_CR36","unstructured":"Li, C., Yates, A., MacAvaney, S., He, B., & Sun, Y. (2020). Parade: Passage representation aggregation for document reranking. arXiv preprint arXiv:2008.09093"},{"key":"9406_CR37","doi-asserted-by":"crossref","unstructured":"Liang, Y., Duan, N., Gong, Y., Wu, N., Guo, F., Qi, W., Gong, M., Shou, L., Jiang, D., Cao, G., Fan, X., Zhang, R., Agrawal, R., Cui, E., Wei, S., Bharti, T., Qiao, Y., Chen, J.H., Wu, W., Liu, S., Yang, F., Campos, D., Majumder, R., & Zhou, M. (2020). XGLUE: A new benchmark dataset for cross-lingual pre-training, understanding and generation. In Proceedings of EMNLP, pp. 6008\u20136018.","DOI":"10.18653\/v1\/2020.emnlp-main.484"},{"key":"9406_CR38","doi-asserted-by":"crossref","unstructured":"Lin, J., Nogueira, R., & Yates, A. (2021). Pretrained transformers for text ranking: BERT and beyond. Synthesis Lectures on Human Language Technologies. Morgan & Claypool Publishers.","DOI":"10.2200\/S01123ED1V01Y202108HLT053"},{"key":"9406_CR39","doi-asserted-by":"crossref","unstructured":"Litschko, R., Glava\u0161, G., Vuli\u0107, I., & Dietz, L. (2019). Evaluating resource-lean cross-lingual embedding models in unsupervised retrieval. In Proceedings of SIGIR, pp. 1109\u20131112.","DOI":"10.1145\/3331184.3331324"},{"key":"9406_CR40","doi-asserted-by":"crossref","unstructured":"Litschko, R., Vuli\u0107, I., Ponzetto, S.P., & Glava\u0161, G. (2021). Evaluating multilingual text encoders for unsupervised cross-lingual retrieval. In Proceedings of ECIR, pp. 342\u2013358.","DOI":"10.1007\/978-3-030-72113-8_23"},{"key":"9406_CR41","doi-asserted-by":"crossref","unstructured":"Liu, F., Vuli\u0107, I., Korhonen, A., & Collier, N. (2021). Fast, effective, and self-supervised: Transforming masked language models into universal lexical and sentence encoders. In Proceedings of EMNLP, pp. 1442\u20131459.","DOI":"10.18653\/v1\/2021.emnlp-main.109"},{"key":"9406_CR42","unstructured":"Liu, Q., Kusner, M.J., & Blunsom, P. (2020). A survey on contextual embeddings. arXiv preprint arXiv:2003.07278"},{"key":"9406_CR43","doi-asserted-by":"crossref","unstructured":"Liu, Q., McCarthy, D., Vuli\u0107, I., & Korhonen, A. (2019). Investigating cross-lingual alignment methods for contextualized embeddings with token-level evaluation. In Proceedings of CoNLL, pp. 33\u201343.","DOI":"10.18653\/v1\/K19-1004"},{"key":"9406_CR44","unstructured":"Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., & Stoyanov, V. (2019). RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692"},{"key":"9406_CR45","doi-asserted-by":"crossref","unstructured":"MacAvaney, S., Cohan, A., & Goharian, N. (2020). SLEDGE-Z: A zero-shot baseline for COVID-19 literature search. In Proceedings of EMNLP, pp. 4171\u20134179.","DOI":"10.18653\/v1\/2020.emnlp-main.341"},{"key":"9406_CR46","doi-asserted-by":"crossref","unstructured":"MacAvaney, S., Soldaini, L., & Goharian, N. (2020). Teaching a new dog old tricks: Resurrecting multilingual retrieval using zero-shot learning. In Proceedings of ECIR, pp. 246\u2013254.","DOI":"10.1007\/978-3-030-45442-5_31"},{"key":"9406_CR47","doi-asserted-by":"crossref","unstructured":"MacAvaney, S., Yates, A., Cohan, A., & Goharian, N. (2019). Cedr: Contextualized embeddings for document ranking. In Proceedings of SIGIR, pp. 1101\u20131104.","DOI":"10.1145\/3331184.3331317"},{"key":"9406_CR48","unstructured":"Mikolov, T., Le, Q.V., & Sutskever, I. (2013). Exploiting similarities among languages for machine translation. CoRR, abs\/1309.4168"},{"key":"9406_CR49","unstructured":"Nguyen, T., Rosenberg, M., Song, X., Gao, J., Tiwary, S., Majumder, R., & Deng, L. (2016). MS MARCO: A human generated machine reading comprehension dataset. Workshop on cognitive computing at NIPS."},{"key":"9406_CR50","unstructured":"Nogueira, R., Yang, W., Cho, K., & Lin, J. (2019). Multi-stage document ranking with BERT. arXiv preprint arXiv:1910.14424"},{"key":"9406_CR51","doi-asserted-by":"crossref","unstructured":"Pires, T., Schlinger, E., & Garrette, D. (2019). How multilingual is multilingual BERT? In Proceedings of ACL, pp. 4996\u20135001.","DOI":"10.18653\/v1\/P19-1493"},{"key":"9406_CR52","doi-asserted-by":"crossref","unstructured":"Ponte, J. M., & Croft, W. B. (1998). A language modeling approach to information retrieval. In Proceedings of SIGIR, pp. 275\u2013281.","DOI":"10.1145\/290941.291008"},{"key":"9406_CR53","doi-asserted-by":"crossref","unstructured":"Ponti, E.M., Glava\u0161, G., Majewska, O., Liu, Q., Vuli\u0107, & I., Korhonen, A. (2020). XCOPA: A multilingual dataset for causal commonsense reasoning. In Proceedings of EMNLP, pp. 2362\u20132376.","DOI":"10.18653\/v1\/2020.emnlp-main.185"},{"key":"9406_CR54","doi-asserted-by":"crossref","unstructured":"Qu, Y., Ding, Y., Liu, J., Liu, K., Ren, R., Zhao, W.X., Dong, D., Wu, H., & Wang, H. (2021). RocketQA: An optimized training approach to dense passage retrieval for open-domain question answering. In Proceedings of NAACL, pp. 5835\u20135847.","DOI":"10.18653\/v1\/2021.naacl-main.466"},{"key":"9406_CR55","unstructured":"Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsupervised multitask learners."},{"key":"9406_CR56","doi-asserted-by":"crossref","unstructured":"Reimers, N., & Gurevych, I. (2019). Sentence-BERT: Sentence embeddings using siamese BERT-networks. In Proceedings of EMNLP, pp. 3973\u20133983.","DOI":"10.18653\/v1\/D19-1410"},{"key":"9406_CR57","doi-asserted-by":"crossref","unstructured":"Reimers, N., & Gurevych, I. (2020). Making monolingual sentence embeddings multilingual using knowledge distillation. In Proceedings of EMNLP, pp. 4512\u20134525.","DOI":"10.18653\/v1\/2020.emnlp-main.365"},{"key":"9406_CR58","doi-asserted-by":"crossref","unstructured":"Robertson, S.E., & Walker, S. (1994). Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. In Proceedings of SIGIR, pp. 232\u2013241.","DOI":"10.1007\/978-1-4471-2099-5_24"},{"key":"9406_CR59","doi-asserted-by":"crossref","unstructured":"Rogers, A., Kovaleva, O., & Rumshisky, A. (2020). A primer in BERTology: What we know about how BERT works. Transactions of the ACL pp. 842\u2013866.","DOI":"10.1162\/tacl_a_00349"},{"key":"9406_CR60","doi-asserted-by":"crossref","unstructured":"Ruder, S., Vuli\u0107, & I., S\u00f8gaard, A. (2019). A survey of cross-lingual word embedding models. Journal of Artificial Intelligence Research, pp. 569\u2013631.","DOI":"10.1613\/jair.1.11640"},{"key":"9406_CR61","unstructured":"Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019). DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108"},{"key":"9406_CR62","doi-asserted-by":"crossref","unstructured":"Sch\u00f6nemann, P. H. (1966). A generalized solution of the orthogonal Procrustes problem. Psychometrika, pp. 1\u201310.","DOI":"10.1007\/BF02289451"},{"key":"9406_CR63","doi-asserted-by":"crossref","unstructured":"Shi, P., Bai, H., & Lin, J. (2020). Cross-lingual training of neural models for document ranking. In Proceedings of EMNLP (Findings), pp. 2768\u20132773.","DOI":"10.18653\/v1\/2020.findings-emnlp.249"},{"key":"9406_CR64","doi-asserted-by":"crossref","unstructured":"Shi, P., Zhang, R., Bai, H., & Lin, J. (2021). Cross-lingual training with dense retrieval for document retrieval. CoRR, abs\/2109.01628","DOI":"10.18653\/v1\/2021.mrl-1.24"},{"key":"9406_CR65","unstructured":"Smith, S. L., Turban, D. H., Hamblin, S., & Hammerla, N. Y. (2017). Offline bilingual word vectors, orthogonal transformations and the inverted softmax. In Proceedings of ICLR."},{"key":"9406_CR66","doi-asserted-by":"crossref","unstructured":"Thakur, N., Reimers, N., Daxenberger, J., & Gurevych, I. (2021). Augmented SBERT: Data augmentation method for improving bi-encoders for pairwise sentence scoring tasks. In Proceedings of NAACL, pp. 296\u2013310.","DOI":"10.18653\/v1\/2021.naacl-main.28"},{"key":"9406_CR67","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., & Polosukhin, I. (2017). Attention is all you need. In Proceedings of NeurIPS, pp. 5998\u20136008."},{"key":"9406_CR68","doi-asserted-by":"crossref","unstructured":"Voorhees, E. (2005). Overview of the trec 2004 robust retrieval track. https:\/\/doi.org\/10.6028\/NIST.SP.500-261","DOI":"10.1145\/1067268.1067272"},{"key":"9406_CR69","doi-asserted-by":"crossref","unstructured":"Vuli\u0107, I., Glava\u0161, G., Reichart, R., & Korhonen, A. (2019). Do we really need fully unsupervised cross-lingual embeddings? In Proceedings of EMNLP, pp. 4406\u20134417.","DOI":"10.18653\/v1\/D19-1449"},{"key":"9406_CR70","doi-asserted-by":"crossref","unstructured":"Vuli\u0107, I., & Moens, M. F. (2015). Monolingual and cross-lingual information retrieval models based on (bilingual) word embeddings. In Proceedings of SIGIR, pp. 363\u2013372.","DOI":"10.1145\/2766462.2767752"},{"key":"9406_CR71","doi-asserted-by":"crossref","unstructured":"Vuli\u0107, I., Ponti, E. M., Litschko, R., Glava\u0161, G., & Korhonen, A. (2020). Probing pretrained language models for lexical semantics. In Proceedings of EMNLP, pp. 7222\u20137240.","DOI":"10.18653\/v1\/2020.emnlp-main.586"},{"key":"9406_CR72","doi-asserted-by":"crossref","unstructured":"Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. R. (2019). GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of ICLR.","DOI":"10.18653\/v1\/W18-5446"},{"key":"9406_CR73","doi-asserted-by":"crossref","unstructured":"Williams, A., Nangia, N., & Bowman, S. A broad-coverage challenge corpus for sentence understanding through inference. In: Proceedings of NAACL, pp. 1112\u20131122 (2018)","DOI":"10.18653\/v1\/N18-1101"},{"key":"9406_CR74","doi-asserted-by":"crossref","unstructured":"Wu, S., & Dredze, M. (2019). Beto, bentz, becas: The surprising cross-lingual effectiveness of bert. In Proceedings of EMNLP, pp. 833\u2013844.","DOI":"10.18653\/v1\/D19-1077"},{"key":"9406_CR75","doi-asserted-by":"crossref","unstructured":"Yang, Y., Abrego, G.H., Yuan, S., Guo, M., Shen, Q., Cer, D., Sung, Y.H., Strope, B., & Kurzweil, R. (2019). Improving multilingual sentence embedding using bi-directional dual encoder with additive margin softmax. In Proceedings of AAAI, pp. 5370\u20135378.","DOI":"10.24963\/ijcai.2019\/746"},{"key":"9406_CR76","doi-asserted-by":"crossref","unstructured":"Yang, Y., Cer, D., Ahmad, A., Guo, M., Law, J., Constant, N., Abrego, G. H., Yuan, S., Tar, C., Sung, Y. H., Strope, B., & Kurzweil, R. (2020). Multilingual universal sentence encoder for semantic retrieval. In Proceedings of ACL: System demonstrations, pp. 87\u201394.","DOI":"10.18653\/v1\/2020.acl-demos.12"},{"key":"9406_CR77","doi-asserted-by":"crossref","unstructured":"Yang, Y., Hernandez\u00a0Abrego, G., Yuan, S., Guo, M., Shen, Q., Cer, D., Sung, Y. H., Strope, B., & Kurzweil, R. (2019). Improving multilingual sentence embedding using bi-directional dual encoder with additive margin softmax. In Proceedings of IJCAI, pp. 5370\u20135378.","DOI":"10.24963\/ijcai.2019\/746"},{"key":"9406_CR78","doi-asserted-by":"crossref","unstructured":"Yu, P., & Allan, J. (2020). A study of neural matching models for cross-lingual IR. In Proceedings of SIGIR, pp. 1637\u20131640.","DOI":"10.1145\/3397271.3401322"},{"key":"9406_CR79","unstructured":"Zaheer, M., Guruganesh, G., Dubey, K. A., Ainslie, J., Alberti, C., Ontanon, S., Pham, P., Ravula, A., Wang, Q., Yang, L., & Ahmed, A. (2020). Big bird: Transformers for longer sequences. In Proceedings of NeurIPS, pp. 17283\u201317297."},{"key":"9406_CR80","doi-asserted-by":"crossref","unstructured":"Zhai, C., & Lafferty, J. A study of smoothing methods for language models applied to information retrieval. ACM Transactions on Information Systems pp. 179\u2013214 (2004)","DOI":"10.1145\/984321.984322"},{"key":"9406_CR81","doi-asserted-by":"crossref","unstructured":"Zhang, X., Ma, X., Shi, P., & Lin, J. (2021). Mr. TyDi: A multi-lingual benchmark for dense retrieval. In Proceedings of the 1st workshop on multilingual representation learning, pp. 127\u2013137.","DOI":"10.18653\/v1\/2021.mrl-1.12"},{"key":"9406_CR82","doi-asserted-by":"crossref","unstructured":"Zhao, M., Zhu, Y., Shareghi, E., Vuli\u0107, I., Reichart, R., Korhonen, A., & Sch\u00fctze, H. (2021). A closer look at few-shot crosslingual transfer: The choice of shots matters. In Proceedings of ACL, pp. 5751\u20135767.","DOI":"10.18653\/v1\/2021.acl-long.447"},{"key":"9406_CR83","doi-asserted-by":"crossref","unstructured":"Zhao, W., Eger, S., Bjerva, J., & Augenstein, I. (2021). Inducing language-agnostic multilingual representations. In Proceedings of *SEM, pp. 229\u2013240.","DOI":"10.18653\/v1\/2021.starsem-1.22"},{"key":"9406_CR84","doi-asserted-by":"crossref","unstructured":"Zhao, W., Glava\u0161, G., Peyrard, M., Gao, Y., West, R., & Eger, S. (2020). On the limitations of cross-lingual encoders as exposed by reference-free machine translation evaluation. In Proceedings of ACL, pp. 1656\u20131671.","DOI":"10.18653\/v1\/2020.acl-main.151"},{"key":"9406_CR85","unstructured":"Ziemski, M., Junczys-Dowmunt, M., & Pouliquen, B. (2016). The United Nations parallel corpus v1.0. In Proceedings of LREC, pp. 3530\u20133534."},{"key":"9406_CR86","doi-asserted-by":"crossref","unstructured":"Zweigenbaum, P., Sharoff, S., & Rapp, R. (2018). Overview of the third bucc shared task: Spotting parallel sentences in comparable corpora. In Proceedings of 11th workshop on building and using comparable corpora, pp. 39\u201342.","DOI":"10.18653\/v1\/W17-2512"}],"container-title":["Information Retrieval Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10791-022-09406-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10791-022-09406-x\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10791-022-09406-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,5,10]],"date-time":"2022-05-10T06:20:31Z","timestamp":1652163631000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10791-022-09406-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,7]]},"references-count":86,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,6]]}},"alternative-id":["9406"],"URL":"https:\/\/doi.org\/10.1007\/s10791-022-09406-x","relation":{},"ISSN":["1386-4564","1573-7659"],"issn-type":[{"value":"1386-4564","type":"print"},{"value":"1573-7659","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,7]]},"assertion":[{"value":"8 July 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 February 2022","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 March 2022","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}