{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T15:19:05Z","timestamp":1778167145355,"version":"3.51.4"},"reference-count":27,"publisher":"Sociedade Brasileira de Computacao - SB","issue":"1","license":[{"start":{"date-parts":[[2026,5,5]],"date-time":"2026-05-05T00:00:00Z","timestamp":1777939200000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JBCS"],"abstract":"<jats:p>Reranking plays a crucial role in improving Information Retrieval (IR) performance, particularly in low-resource languages, such as Portuguese. In this study, we evaluate different reranking strategies for Portuguese IR, comparing multilingual and Portuguese-specific models, as well as not-so-large language models and large language models (LLMs). We assess the performance of BM25 combined with ptT5 fine-tuned on multilingual and Brazilian Portuguese datasets, alongside multilingual state-of-the-art rerankers (BGE m3) and LLM as rerankers RankGPT (GPT-4) and Sabi\u00e1 3, a Portuguese-specific LLM. Additionally, we introduce a novel dynamic In-Context Learning (DICL) prompting strategy to enhance LLM performance. Experiments conducted on the Quati and Pir\u00e1 2.0 datasets show that fine-tuning on native Brazilian Portuguese data significantly improves retrieval effectiveness by up to 5 p.p. in nDCG compared to using translated multilingual datasets. Two fine-tuning approaches were tested: a binary classification strategy with \u2018true\u2019 and \u2018false\u2019 tokens and a relevance score-based training, both outperforming models fine-tuned on translated multilingual data. RankGPT achieved the best overall results, yet Sabi\u00e1 3 demonstrated competitive performance, particularly on queries related to sociocultural aspects. The DICL strategy further improved the results of both LLMs, significantly boosting their MRR@10. These findings highlight the importance of language-specific training and suggest that not-so-large language models can be viable alternatives for reranking tasks in Portuguese IR.<\/jats:p>","DOI":"10.5753\/jbcs.2026.5659","type":"journal-article","created":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T14:47:28Z","timestamp":1778165248000},"page":"1207-1220","source":"Crossref","is-referenced-by-count":0,"title":["Evaluating Reranking Strategies for Portuguese Information Retrieval: Fine-Tuning, LLMs, and Sociocultural Aspects"],"prefix":"10.5753","volume":"32","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7279-4546","authenticated-orcid":false,"given":"Renato Okabayashi","family":"Miyaji","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8743-4244","authenticated-orcid":false,"given":"Pedro Luiz Pizzigatti","family":"Corr\u00eaa","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"3742","published-online":{"date-parts":[[2026,5,5]]},"reference":[{"key":"1","unstructured":"Abonizio, H., Almeida, T., Laitz, T., Junior, R., Bon\u00e1s, G., Nogueira, R., and Pires, R. (2024). Sabi\u00e1-3 technical report. <i>ArXiv<\/i>."},{"key":"2","unstructured":"Bonifacio, L., Campiotti, I., Lotufo, R., and Nogueira, R. (2021). mmarco: A multilingual version of ms marco passage ranking dataset. <i>CoRR<\/i>, 15413(1). DOI: <a href=\"https:\/\/doi.org\/CoRR abs\/2108.13897\">CoRR abs\/2108.13897<\/a>."},{"key":"3","unstructured":"Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. (2020). Language models are few-shot learners. In <i>Proceedings of the 34th International Conference on Neural Information Processing Systems<\/i>, NIPS '20, Red Hook, NY, USA. Curran Associates Inc."},{"key":"4","doi-asserted-by":"crossref","unstructured":"Bueno, M., Oliveira, E., Nogueira, R., Lotufo, R., and Pereira, J. (2024). Quati: A brazilian portuguese information retrieval dataset from native speakers. <i>Proceedings of the XV Brazilian Symposium on Information Technology and Human Language (STIL)<\/i>, 1.","DOI":"10.5753\/stil.2024.245426"},{"key":"5","unstructured":"Carmo, D., Piau, M., Campiotti, I., Nogueira, R., and Lotufo, R. (2020). Ptt5: Pretraining and validating the t5 model on brazilian portuguese data. <i>ArXiv<\/i>, 1."},{"key":"6","doi-asserted-by":"crossref","unstructured":"Caseli, H. and Nunes, M. (2023). <i>Processamento de Linguagem Natural: Conceitos, T\u00e9cnicas e Aplica\u00e7\u00f5es em Portugu\u00eas<\/i>. Brasileiras - Processamento de Linguagem Natural.","DOI":"10.5753\/wit.2023.229504"},{"key":"7","doi-asserted-by":"crossref","unstructured":"Chen, J., Xiao, S., Zhang, P., Luo, K., Lian, D., and Liu, Z. (2024). M3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. <i>Findings of the Association for Computational Linguistics: ACL 2024<\/i>.","DOI":"10.18653\/v1\/2024.findings-acl.137"},{"key":"8","doi-asserted-by":"crossref","unstructured":"Guo, F., Li, W., Zhuang, H., Luo, Y., Li, Y., Yan, L., Zhu, Q., and Zhang, Y. (2025). Mcranker: Generating diverse criteria on-the-fly to improve pointwise llm rankers. <i>Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining<\/i>.","DOI":"10.1145\/3701551.3703583"},{"key":"9","doi-asserted-by":"crossref","unstructured":"Howard, J. and Ruder, S. (2018). Universal language model fine-tuning for text classification. <i>Annual Meeting of the Association for Computational Linguistics<\/i>.","DOI":"10.18653\/v1\/P18-1031"},{"key":"10","doi-asserted-by":"crossref","unstructured":"Jones, K., Walker, S., and Robertson, S. (2000). A probabilistic model of information retrieval: development and comparative experiments. <i>Information processing & management<\/i>, 36(6).","DOI":"10.1016\/S0306-4573(00)00015-7"},{"key":"11","doi-asserted-by":"crossref","unstructured":"Laitz, T., Papakostas, K., Lotufo, R., and Nogueira, R. (2025). Inranker: Distilled rankers for zero-shot information retrieval. <i>Intelligent Systems. BRACIS 2024. Lecture Notes in Computer Science<\/i>, 15413(1). DOI: <a href=\"https:\/\/doi.org\/10.1007\/978-3-031-79032-4_10\">10.1007\/978-3-031-79032-4_10<\/a>.","DOI":"10.1007\/978-3-031-79032-4_10"},{"key":"12","unstructured":"Lewis, L., Yang, Y., Rose, T., and Li, F. (2004). Rcv1: A new benchmark collection for text categorization research. <i>Journal of Machine Learning Research<\/i>, 5(1)."},{"key":"13","unstructured":"Ma, X., Zhang, X., Pradeep, R., and Lin, J. (2023). Zero-shot listwise document reranking with a large language model. <i>CoRR<\/i>."},{"key":"14","unstructured":"Maritaca AI (2025). Maritaca ai documentation - models. [<a href=\"https:\/\/docs.maritaca.ai\/pt\/modelos\">link<\/a>]. Accessed on 15 January 2025."},{"key":"15","unstructured":"Nguyen, T., Rosenberg, M., Song, X., Gao, J., Tiwary, S., Majumder, R., and Deng, L. (2016). Ms marco: A human generated machine reading comprehension dataset. <i>Annual Conference on Neural Information Processing Systems (NIPS)<\/i>, 1."},{"key":"16","doi-asserted-by":"crossref","unstructured":"Nogueira, R., Jiang, Z., Pradeep, R., and Lin, J. (2020). Document ranking with a pretrained sequence-to-sequence model. <i>Empirical Methods in Natural Language Processing<\/i>.","DOI":"10.18653\/v1\/2020.findings-emnlp.63"},{"key":"17","unstructured":"Nogueira, R., Yang, W., Cho, K., and Lin, J. (2019). Multi-stage document ranking with bert. <i>ArXiv<\/i>."},{"key":"18","unstructured":"Oliveira, L., Romeu, R., and Moreira, V. (2021). Regis: A test collection for geoscientific documents in portuguese. <i>Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval<\/i>."},{"key":"19","unstructured":"OpenAI (2025). Openai documentation - pricing. [<a href=\"https:\/\/platform.openai.com\/docs\/pricing\">link<\/a>]. Accessed on 15 January 2025."},{"key":"20","doi-asserted-by":"crossref","unstructured":"Paschoal, A., Pirozelli, P., Freire, V., Delgado, K., Peres, S., Jos\u00e9, M., Nakasato, F., Oliveira, A., Brand\u00e3o, A., Costa, A., and Cozman, F. (2021). Pir\u00e1: A bilingual portuguese-english dataset for question-answering about the ocean. <i>Proceedings of the 30th ACM International Conference on Information & Knowledge Management<\/i>.","DOI":"10.1145\/3459637.3482012"},{"key":"21","doi-asserted-by":"crossref","unstructured":"Pirozelli, P., Jos\u00e9, M., Silveira, I., Nakasato, F., Peres, S., Brand\u00e3o, A., Costa, A., and Cozman, F. (2024). Benchmarks for pir\u00e1 2.0, a reading comprehension dataset about the ocean, the brazilian coast, and climate change. <i>Data Intelligence<\/i>, 1(6):29-63.","DOI":"10.1162\/dint_a_00245"},{"key":"22","doi-asserted-by":"crossref","unstructured":"Qin, Z., Jagerman, R., Hui, K., Zhuang, H., Wu, J., Yan, L., Shen, J., Liu, J., Liu, J., Metzler, D., Wang, X., and Bendersky, M. (2024). Large language models are effective text rankers with pairwise ranking prompting. <i>Findings of the Association for Computational Linguistics: NAACL 2024<\/i>.","DOI":"10.18653\/v1\/2024.findings-naacl.97"},{"key":"23","doi-asserted-by":"crossref","unstructured":"Sachan, D., Lewis, M., Joshi, M., Aghajanyan, A., Yih, W., Pineau, J., and Zettlemoyer, L. (2022). Improving passage retrieval with zero-shot question generation. <i>Empirical Methods in Natural Language Processing<\/i>.","DOI":"10.18653\/v1\/2022.emnlp-main.249"},{"key":"24","unstructured":"Sentence-Transformers (2021). all-minilm-l6-v2. [<a href=\"https:\/\/huggingface.co\/sentence-transformers\/all-MiniLM-L6-v2\">link<\/a>]. Pretrained sentence embedding model. Fine-tuned on 1B+ sentence pairs."},{"key":"25","doi-asserted-by":"crossref","unstructured":"Sun, W., Yan, L., Ma, X., Wang, S., Ren, P., Chen, Z., Yin, D., and Ren, Z. (2023). Is chatgpt good at search? investigating large language models as re-ranking agents. <i>Empirical Methods in Natural Language Processing<\/i>.","DOI":"10.18653\/v1\/2023.emnlp-main.923"},{"key":"26","doi-asserted-by":"crossref","unstructured":"Xue, L., Constant, N., Roberts, A., Kale, M., Al-Rfou, R., Siddhant, A., Barua, A., and Raffel, C. (2021). mt5: A massively multilingual pre-trained text-to-text transformer. <i>Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies<\/i>, 1.","DOI":"10.18653\/v1\/2021.naacl-main.41"},{"key":"27","unstructured":"Zhu, Y., Yuan, H., Wang, S., Liu, S., Liu, W., Deng, C., Chen, H., Liu, Z., Dou, Z., and Wen, J. (2024). Large language models for information retrieval: A survey. <i>ArXiv<\/i>."}],"container-title":["Journal of the Brazilian Computer Society"],"original-title":[],"link":[{"URL":"https:\/\/journals-sol.sbc.org.br\/index.php\/jbcs\/article\/download\/5659\/3983","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals-sol.sbc.org.br\/index.php\/jbcs\/article\/download\/5659\/3983","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T14:47:45Z","timestamp":1778165265000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals-sol.sbc.org.br\/index.php\/jbcs\/article\/view\/5659"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,5]]},"references-count":27,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,1,20]]}},"URL":"https:\/\/doi.org\/10.5753\/jbcs.2026.5659","relation":{},"ISSN":["1678-4804"],"issn-type":[{"value":"1678-4804","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,5]]}}}