{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,8]],"date-time":"2026-02-08T04:09:21Z","timestamp":1770523761177,"version":"3.49.0"},"publisher-location":"New York, NY, USA","reference-count":39,"publisher":"ACM","funder":[{"name":"Bundesministerium f\u00fcr Bildung und Forschung","award":["011S24077B"],"award-info":[{"award-number":["011S24077B"]}]},{"name":"Bundesministerium f\u00fcr Bildung und Forschung","award":["ScaDS.AI"],"award-info":[{"award-number":["ScaDS.AI"]}]},{"DOI":"10.13039\/501100006374","name":"HORIZON EUROPE European Research Council","doi-asserted-by":"publisher","award":["GA 101070014"],"award-info":[{"award-number":["GA 101070014"]}],"id":[{"id":"10.13039\/501100006374","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,7,18]]},"DOI":"10.1145\/3731120.3744594","type":"proceedings-article","created":{"date-parts":[[2025,7,18]],"date-time":"2025-07-18T13:34:06Z","timestamp":1752845646000},"page":"275-285","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Learning Effective Representations for Retrieval Using Self-Distillation with Adaptive Relevance Margins"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5707-3751","authenticated-orcid":false,"given":"Lukas","family":"Gienapp","sequence":"first","affiliation":[{"name":"University of Kassel, Kassel, Germany, ScaDS.AI, Leipzig, Germany, and hessian.AI, Darmstadt, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6803-1223","authenticated-orcid":false,"given":"Niklas","family":"Deckers","sequence":"additional","affiliation":[{"name":"University of Kassel, Kassel, Germany and hessian.AI, Darmstadt, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2451-0665","authenticated-orcid":false,"given":"Martin","family":"Potthast","sequence":"additional","affiliation":[{"name":"University of Kassel, Kassel, Germany, ScaDS.AI, Leipzig, Germany, and hessian.AI, Darmstadt, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9578-7157","authenticated-orcid":false,"given":"Harrisen","family":"Scells","sequence":"additional","affiliation":[{"name":"University of T\u00fcbingen, T\u00fcbingen, Germany"}]}],"member":"320","published-online":{"date-parts":[[2025,7,18]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3511808.3557343"},{"key":"e_1_3_2_1_2_1","volume-title":"Pre-training Tasks for Embedding-based Large-scale Retrieval. In 8th International Conference on Learning Representations, ICLR 2020","author":"Chang Wei-Cheng","year":"2020","unstructured":"Wei-Cheng Chang, Felix X. Yu, Yin-Wen Chang, Yiming Yang, and Sanjiv Kumar. 2020. Pre-training Tasks for Embedding-based Large-scale Retrieval. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.6028\/NIST.SP.1266.deep-overview"},{"key":"e_1_3_2_1_4_1","volume-title":"Overview of the TREC 2019 deep learning track. (2020","author":"Craswell Nick","year":"2003","unstructured":"Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M. Voorhees. 2020b. Overview of the TREC 2019 deep learning track. (2020). arXiv:2003.07820"},{"key":"e_1_3_2_1_5_1","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019","volume":"1","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), J. Burstein, C. Doran, and T. Solorio, (Eds.). ACL, 4171-4186."},{"key":"e_1_3_2_1_6_1","unstructured":"William Falcon and The PyTorch Lightning team. 2019. PyTorch Lightning."},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3463098"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.75"},{"key":"e_1_3_2_1_9_1","volume-title":"ECIR 2021, Virtual Event, March 28 - April 1, 2021, Proceedings, Part I, (Lecture Notes in Computer Science","volume":"160","author":"Gao Luyu","year":"2021","unstructured":"Luyu Gao, Zhuyun Dai, Tongfei Chen, Zhen Fan, Benjamin Van Durme, and Jamie Callan. 2021. Complement Lexical Retrieval Model with Semantic Residual Embeddings. In Advances in Information Retrieval - 43rd European Conference on IR Research, ECIR 2021, Virtual Event, March 28 - April 1, 2021, Proceedings, Part I, (Lecture Notes in Computer Science, Vol. 12656), D. Hiemstra, M.-F. Moens, J. Mothe, R. Perego, M. Potthast, and F. Sebastiani, (Eds.). Springer, 146-160."},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-021-01453-z"},{"key":"e_1_3_2_1_11_1","unstructured":"Mai Lan Ha and Volker Blanz. 2021. Deep Ranking with Adaptive Margin Triplet Loss. (2021). arXiv:2107.06187"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.26615\/978-954-452-056-4_050"},{"key":"e_1_3_2_1_13_1","unstructured":"Geoffrey E. Hinton Oriol Vinyals and Jeffrey Dean. 2015. Distilling the Knowledge in a Neural Network. (2015). arXiv:1503.02531"},{"key":"e_1_3_2_1_14_1","unstructured":"Sebastian Hofst\u00e4tter Sophia Althammer Michael Schr\u00f6der Mete Sertkan and Allan Hanbury. 2020. Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation. (2020). arXiv:2010.02666"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462891"},{"key":"e_1_3_2_1_16_1","unstructured":"Chengming Hu Xuan Li Dan Liu Haolun Wu Xi Chen Ju Wang and Xue Liu. 2023. Teacher-Student Architecture for Knowledge Distillation: A Survey. (2023). arXiv:2308.04268"},{"key":"e_1_3_2_1_17_1","article-title":"Unsupervised Dense Information Retrieval with Contrastive","volume":"2022","author":"Izacard Gautier","year":"2022","unstructured":"Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2022. Unsupervised Dense Information Retrieval with Contrastive Learning. Trans. Mach. Learn. Res., Vol. 2022 (2022).","journal-title":"Learning. Trans. Mach. Learn. Res."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/TBDATA.2019.2921572"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3626772.3657862"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2020.EMNLP-MAIN.550"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401075"},{"key":"e_1_3_2_1_22_1","volume-title":"Boosting Few-Shot Learning With Adaptive Margin Loss. In 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020","author":"Li Aoxue","year":"2020","unstructured":"Aoxue Li, Weiran Huang, Xu Lan, Jiashi Feng, Zhenguo Li, and Liwei Wang. 2020. Boosting Few-Shot Learning With Adaptive Margin Loss. In 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020. Computer Vision Foundation \/ IEEE, 12573-12581."},{"key":"e_1_3_2_1_23_1","volume-title":"RoBERTa: A Robustly Optimized BERT Pretraining Approach. CoRR","author":"Liu Yinhan","year":"2019","unstructured":"Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. CoRR, Vol. abs\/1907.11692 (2019). arxiv:1907.11692"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.56"},{"key":"e_1_3_2_1_25_1","unstructured":"Tri Nguyen Mir Rosenberg Xia Song Jianfeng Gao Saurabh Tiwary Rangan Majumder and Li Deng. 2016. MS MARCO: A Human Generated MAchine Reading COmprehension Dataset. In Proceedings of the Workshop on Cognitive Computation: Integrating neural and symbolic approaches 2016 co-located with the 30th Annual Conference on Neural Information Processing Systems (NIPS 2016) Barcelona Spain December 9 2016 (CEUR Workshop Proceedings Vol. 1773) T. R. Besold A. Bordes A. S. d'Avila Garcez and G. Wayne (Eds.). CEUR-WS.org."},{"key":"e_1_3_2_1_26_1","unstructured":"Rodrigo Frassetto Nogueira Wei Yang Kyunghyun Cho and Jimmy Lin. 2019. Multi-Stage Document Ranking with BERT. (2019). arXiv:1910.14424"},{"key":"e_1_3_2_1_27_1","first-page":"8024","volume-title":"High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019","author":"Paszke Adam","year":"2019","unstructured":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas K\u00f6pf, Edward Z. Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, H. M. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alch\u00e9-Buc, E. B. Fox, and R. Garnett, (Eds.). 8024-8035."},{"key":"e_1_3_2_1_28_1","unstructured":"Victor Sanh Lysandre Debut Julien Chaumond and Thomas Wolf. 2019. DistilBERT a distilled version of BERT: smaller faster cheaper and lighter. (2019). arXiv:1910.01108"},{"key":"e_1_3_2_1_29_1","first-page":"2825","volume-title":"Recycle: Green Information Retrieval Research. In SIGIR '22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Scells Harrisen","year":"2022","unstructured":"Harrisen Scells, Shengyao Zhuang, and Guido Zuccon. 2022. Reduce, Reuse, Recycle: Green Information Retrieval Research. In SIGIR '22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, July 11 - 15, 2022, E. Amig\u00f3, P. Castells, J. Gonzalo, B. Carterette, J. S. Culpepper, and G. Kazai, (Eds.). ACM, 2825-2837."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298682"},{"key":"e_1_3_2_1_31_1","volume-title":"AUC-CL: A Batchsize-Robust Framework for Self-Supervised Contrastive Representation Learning. In The Twelfth International Conference on Learning Representations, ICLR 2024","author":"Sharma Rohan","year":"2024","unstructured":"Rohan Sharma, Kaiyi Ji, Zhiqiang Xu, and Changyou Chen. 2024. AUC-CL: A Batchsize-Robust Framework for Self-Supervised Contrastive Representation Learning. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net. https:\/\/openreview.net\/forum?id=YgMdDQB09U"},{"key":"e_1_3_2_1_32_1","volume-title":"MPNet: Masked and Permuted Pre-training for Language Understanding. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020","author":"Song Kaitao","year":"2020","unstructured":"Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020. MPNet: Masked and Permuted Pre-training for Language Understanding. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, H. Larochelle, M. Ranzato, R. Hadsell, M.-F. Balcan, and H.-T. Lin, (Eds.)."},{"key":"e_1_3_2_1_33_1","volume-title":"Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Datasets and Benchmarks 2021","author":"Thakur Nandan","year":"2021","unstructured":"Nandan Thakur, Nils Reimers, Andreas R\u00fcckl\u00e9, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models. In Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks 1, NeurIPS Datasets and Benchmarks 2021, December 2021, virtual, Joaquin Vanschoren and Sai-Kit Yeung, (Eds.). https:\/\/datasets-benchmarks-proceedings.neurips.cc\/paper\/2021\/hash\/65b9eea6e1cc6bb9f0cd2a47751a186f-Abstract-round2.html"},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3539618.3591915"},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-demos.6"},{"key":"e_1_3_2_1_36_1","volume-title":"Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval. In 9th International Conference on Learning Representations, ICLR 2021","author":"Xiong Lee","year":"2021","unstructured":"Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N. Bennett, Junaid Ahmed, and Arnold Overwijk. 2021. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net."},{"key":"e_1_3_2_1_37_1","first-page":"1503","volume-title":"Optimizing Dense Retrieval Model Training with Hard Negatives. In SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Zhan Jingtao","year":"2021","unstructured":"Jingtao Zhan, Jiaxin Mao, Yiqun Liu, Jiafeng Guo, Min Zhang, and Shaoping Ma. 2021. Optimizing Dense Retrieval Model Training with Hard Negatives. In SIGIR '21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, Canada, July 11-15, 2021, F. Diaz, C. Shah, T. Suel, P. Castells, R. Jones, and T. Sakai, (Eds.). ACM, 1503-1512."},{"key":"e_1_3_2_1_38_1","unstructured":"Wayne Xin Zhao Jing Liu Ruiyang Ren and Ji-Rong Wen. 2022. Dense Text Retrieval based on Pretrained Language Models: A Survey. (2022). arXiv:2211.14876"},{"key":"e_1_3_2_1_39_1","unstructured":"Yang Zhou Zhanhao He Keyu Lu Guanhong Wang and Gaoang Wang. 2022. Preserve Pre-trained Knowledge: Transfer Learning With Self-Distillation For Action Recognition. (2022). arXiv:2205.00506"}],"event":{"name":"ICTIR '25: International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval","location":"Padua Italy","acronym":"ICTIR '25","sponsor":["SIGIR ACM Special Interest Group on Information Retrieval"]},"container-title":["Proceedings of the 2025 International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR)"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3731120.3744594","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T13:17:26Z","timestamp":1755868646000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3731120.3744594"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,18]]},"references-count":39,"alternative-id":["10.1145\/3731120.3744594","10.1145\/3731120"],"URL":"https:\/\/doi.org\/10.1145\/3731120.3744594","relation":{},"subject":[],"published":{"date-parts":[[2025,7,18]]},"assertion":[{"value":"2025-07-18","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}