{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T00:59:54Z","timestamp":1774400394382,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":47,"publisher":"ACM","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,7,13]]},"DOI":"10.1145\/3726302.3730225","type":"proceedings-article","created":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T01:38:52Z","timestamp":1752457132000},"page":"2679-2684","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["Scaling Sparse and Dense Retrieval in Decoder-Only LLMs"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-2699-8460","authenticated-orcid":false,"given":"Hansi","family":"Zeng","sequence":"first","affiliation":[{"name":"University of Massachusetts Amherst, Amherst, MA, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2280-8759","authenticated-orcid":false,"given":"Julian","family":"Killingback","sequence":"additional","affiliation":[{"name":"University of Massachusetts Amherst, Amherst, MA, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0800-3340","authenticated-orcid":false,"given":"Hamed","family":"Zamani","sequence":"additional","affiliation":[{"name":"University of Massachusetts Amherst, Amherst, MA, USA"}]}],"member":"320","published-online":{"date-parts":[[2025,7,13]]},"reference":[{"key":"e_1_3_2_1_1_1","unstructured":"Payal Bajaj Daniel Campos Nick Craswell Li Deng Jianfeng Gao Xiaodong Liu Rangan Majumder Andrew McNamara Bhaskar Mitra Tri Nguyen et al. 2016. Ms marco: A human generated machine reading comprehension dataset. arXiv preprint arXiv:1611.09268 (2016)."},{"key":"e_1_3_2_1_2_1","volume-title":"First Conference on Language Modeling. https:\/\/openreview.net\/forum?id=IW1PR7vEBf","author":"BehnamGhader Parishad","year":"2024","unstructured":"Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy. 2024. LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders. In First Conference on Language Modeling. https:\/\/openreview.net\/forum?id=IW1PR7vEBf"},{"key":"e_1_3_2_1_3_1","unstructured":"Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell Sandhini Agarwal Ariel Herbert-Voss Gretchen Krueger Tom Henighan Rewon Child Aditya Ramesh Daniel M. Ziegler Jeffrey Wu Clemens Winter Christopher Hesse Mark Chen Eric Sigler Mateusz Litwin Scott Gray Benjamin Chess Jack Clark Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever and Dario Amodei. 2020. Language Models are Few-Shot Learners. arXiv:2005.14165 [cs.CL] https:\/\/arxiv.org\/abs\/2005.14165"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3511808.3557456"},{"key":"e_1_3_2_1_5_1","volume-title":"Overview of the TREC 2020 deep learning track. arXiv:2102","author":"Craswell Nick","year":"2021","unstructured":"Nick Craswell, Bhaskar Mitra, Emine Yilmaz, and Daniel Campos. 2021. Overview of the TREC 2020 deep learning track. arXiv:2102.07662 [cs.IR] https:\/\/arxiv.org\/abs\/2102.07662"},{"key":"e_1_3_2_1_6_1","volume-title":"Overview of the TREC 2019 deep learning track. arXiv:2003","author":"Craswell Nick","year":"2003","unstructured":"Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M. Voorhees. 2020. Overview of the TREC 2019 deep learning track. arXiv:2003.07820 [cs.IR] https:\/\/arxiv.org\/abs\/2003.07820"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401204"},{"key":"e_1_3_2_1_8_1","volume-title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In North American","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In North American Chapter of the Association for Computational Linguistics. https:\/\/api.semanticscholar.org\/CorpusID:52967399"},{"key":"e_1_3_2_1_9_1","volume-title":"Mistral-SPLADE: LLMs for better Learned Sparse Retrieval. ArXiv abs\/2408.11119","author":"Doshi Meet","year":"2024","unstructured":"Meet Doshi, Vishwajeet Kumar, Rudra Murthy, P Vignesh, and Jaydeep Sen. 2024. Mistral-SPLADE: LLMs for better Learned Sparse Retrieval. ArXiv abs\/2408.11119 (2024). https:\/\/api.semanticscholar.org\/CorpusID:271915981"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3626772.3657743"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3477495.3531857"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3463098"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-long.203"},{"key":"e_1_3_2_1_14_1","volume-title":"Distilling the Knowledge in a Neural Network. ArXiv abs\/1503.02531","author":"Hinton Geoffrey E.","year":"2015","unstructured":"Geoffrey E. Hinton, Oriol Vinyals, and Jeffrey Dean. 2015. Distilling the Knowledge in a Neural Network. ArXiv abs\/1503.02531 (2015). https:\/\/api.semanticscholar.org\/CorpusID:7200347"},{"key":"e_1_3_2_1_15_1","volume-title":"Proceedings of the 36th International Conference on Neural Information Processing Systems","author":"Hoffmann Jordan","unstructured":"Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Oriol Vinyals, Jack W. Rae, and Laurent Sifre. 2022. Training computeoptimal large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems (New Orleans, LA, USA) (NIPS '22). Curran Associates Inc., Red Hook, NY, USA, Article 2176, 15 pages."},{"key":"e_1_3_2_1_16_1","volume-title":"Improving Efficient Neural Ranking Models with Cross- Architecture Knowledge Distillation. ArXiv abs\/2010.02666","author":"Hofst\u00e4tter Sebastian","year":"2020","unstructured":"Sebastian Hofst\u00e4tter, Sophia Althammer, Michael Schr\u00f6der, Mete Sertkan, and Allan Hanbury. 2020. Improving Efficient Neural Ranking Models with Cross- Architecture Knowledge Distillation. ArXiv abs\/2010.02666 (2020). https:\/\/api.semanticscholar.org\/CorpusID:222141041"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462891"},{"key":"e_1_3_2_1_18_1","volume-title":"LoRA: Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=nZeVKeeFYf9","author":"Hu Edward J","year":"2022","unstructured":"Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=nZeVKeeFYf9"},{"key":"e_1_3_2_1_19_1","unstructured":"Suyuan Huang Chao Zhang Yuanyuan Wu Haoxin Zhang Yuan Wang Maolin Wang Shaosheng Cao Tong Xu Xiangyu Zhao Zengchang Qin Yan Gao Yunhan Bai Jun Fan Yao Hu and Enhong Chen. 2024. ScalingNote: Scaling up Retrievers with Large Language Models for Real-World Dense Retrieval. arXiv:2411.15766 [cs.IR] https:\/\/arxiv.org\/abs\/2411.15766"},{"key":"e_1_3_2_1_20_1","volume-title":"Scaling Laws for Downstream Task Performance in Machine Translation. In The Thirteenth International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=vPOMTkmSiu","author":"Isik Berivan","year":"2025","unstructured":"Berivan Isik, Natalia Ponomareva, Hussein Hazimeh, Dimitris Paparas, Sergei Vassilvitskii, and Sanmi Koyejo. 2025. Scaling Laws for Downstream Task Performance in Machine Translation. In The Thirteenth International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=vPOMTkmSiu"},{"key":"e_1_3_2_1_21_1","volume-title":"Query Expansion by Prompting Large Language Models. ArXiv abs\/2305.03653","author":"Jagerman Rolf","year":"2023","unstructured":"Rolf Jagerman, Honglei Zhuang, Zhen Qin, Xuanhui Wang, and Michael Bendersky. 2023. Query Expansion by Prompting Large Language Models. ArXiv abs\/2305.03653 (2023). https:\/\/api.semanticscholar.org\/CorpusID:258546701"},{"key":"e_1_3_2_1_22_1","unstructured":"Jeff Johnson Matthijs Douze and Herv\u00e9 J\u00e9gou. 2017. Billion-scale similarity search with GPUs. arXiv:1702.08734 [cs.CV] https:\/\/arxiv.org\/abs\/1702.08734"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.550"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-emnlp.423"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.repl4nlp-1.17"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.acl-long.191"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462869"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3626772.3657951"},{"key":"e_1_3_2_1_29_1","unstructured":"Llama Team AI @ Meta. 2024. The Llama 3 Herd of Models. arXiv:2407.21783 [cs.AI] https:\/\/arxiv.org\/abs\/2407.21783"},{"key":"e_1_3_2_1_30_1","volume-title":"International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=SygpC6Ntvr","author":"Paria Biswajit","year":"2020","unstructured":"Biswajit Paria, Chih-Kuan Yeh, Ian E.H. Yen, Ning Xu, Pradeep Ravikumar, and Barnab\u00e1s P\u00f3czos. 2020. Minimizing FLOPs to Learn Efficient Sparse Representations. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=SygpC6Ntvr"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.224"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.naacl-main.272"},{"key":"e_1_3_2_1_33_1","volume-title":"Repetition Improves Language Model Embeddings. In The Thirteenth International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=Ahlrf2HGJR","author":"Springer Jacob Mitchell","year":"2025","unstructured":"Jacob Mitchell Springer, Suhas Kotha, Daniel Fried, Graham Neubig, and Aditi Raghunathan. 2025. Repetition Improves Language Model Embeddings. In The Thirteenth International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=Ahlrf2HGJR"},{"key":"e_1_3_2_1_34_1","volume-title":"LLMs are Also Effective Embedding Models: An Indepth Overview. ArXiv abs\/2412.12591","author":"Tao Chongyang","year":"2024","unstructured":"Chongyang Tao, Tao Shen, Shen Gao, Junshuo Zhang, Zhen Li, Zhengwei Tao, and Shuai Ma. 2024. LLMs are Also Effective Embedding Models: An Indepth Overview. ArXiv abs\/2412.12591 (2024). https:\/\/api.semanticscholar.org\/CorpusID:274789267"},{"key":"e_1_3_2_1_35_1","volume-title":"Proceedings of the Human Language Technology Conference of the NAACL, Main Conference, Robert C","author":"Tao Tao","unstructured":"Tao Tao, Xuanhui Wang, Qiaozhu Mei, and ChengXiang Zhai. 2006. Language Model Information Retrieval with Document Expansion. In Proceedings of the Human Language Technology Conference of the NAACL, Main Conference, Robert C. Moore, Jeff Bilmes, Jennifer Chu-Carroll, and Mark Sanderson (Eds.). Association for Computational Linguistics, New York City, USA, 407-414. https:\/\/aclanthology.org\/N06-1052\/"},{"key":"e_1_3_2_1_36_1","unstructured":"Nandan Thakur Nils Reimers Andreas R\u00fcckl\u00e9 Abhishek Srivastava and Iryna Gurevych. 2021. BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). https:\/\/openreview.net\/forum?id=wCu6T5xFjeJ"},{"key":"e_1_3_2_1_37_1","unstructured":"Hugo Touvron et al. 2023. Llama 2: Open Foundation and Fine-Tuned Chat Models. ArXiv abs\/2307.09288 (2023). https:\/\/api.semanticscholar.org\/CorpusID:259950998"},{"key":"e_1_3_2_1_38_1","unstructured":"Yunli Wang Zixuan Yang Zhen Zhang Zhiqiang Wang Jian Yang Shiyang Wen Peng Jiang and Kun Gai. 2024. Scaling Laws for Online Advertisement Retrieval. arXiv:2411.13322 [cs.IR] https:\/\/arxiv.org\/abs\/2411.13322"},{"key":"e_1_3_2_1_39_1","volume-title":"Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus.","author":"Wei Jason","year":"2022","unstructured":"Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022. Emergent Abilities of Large Language Models. Transactions on Machine Learning Research (2022). https:\/\/openreview.net\/forum?id=yzkSU5zdwD Survey Certification."},{"key":"e_1_3_2_1_40_1","volume-title":"Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=zeFrfgyZln","author":"Xiong Lee","year":"2021","unstructured":"Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N. Bennett, Junaid Ahmed, and Arnold Overwijk. 2021. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=zeFrfgyZln"},{"key":"e_1_3_2_1_41_1","volume-title":"CSPLADE: Learned Sparse Retrieval with Causal Language Models. https:\/\/api.semanticscholar.org\/CorpusID:277786967","author":"Xu Zhichao","year":"2025","unstructured":"Zhichao Xu, Aosong Feng, Yijun Tian, Haibo Ding, and Lin Leee Cheong. 2025. CSPLADE: Learned Sparse Retrieval with Causal Language Models. https:\/\/api.semanticscholar.org\/CorpusID:277786967"},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3269206.3271800"},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3539618.3591626"},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3477495.3531791"},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462880"},{"key":"e_1_3_2_1_46_1","volume-title":"Model and Finetuning Method. In The Twelfth International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=5HCnKDeTws","author":"Zhang Biao","year":"2024","unstructured":"Biao Zhang, Zhongtao Liu, Colin Cherry, and Orhan Firat. 2024. When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method. In The Twelfth International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=5HCnKDeTws"},{"key":"e_1_3_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/1132956.1132959"}],"event":{"name":"SIGIR '25: The 48th International ACM SIGIR Conference on Research and Development in Information Retrieval","location":"Padua Italy","acronym":"SIGIR '25","sponsor":["SIGIR ACM Special Interest Group on Information Retrieval"]},"container-title":["Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3726302.3730225","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T12:08:17Z","timestamp":1755864497000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3726302.3730225"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,13]]},"references-count":47,"alternative-id":["10.1145\/3726302.3730225","10.1145\/3726302"],"URL":"https:\/\/doi.org\/10.1145\/3726302.3730225","relation":{},"subject":[],"published":{"date-parts":[[2025,7,13]]},"assertion":[{"value":"2025-07-13","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}