{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,15]],"date-time":"2026-07-15T05:22:52Z","timestamp":1784092972779,"version":"3.55.0"},"publisher-location":"New York, NY, USA","reference-count":48,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,7,6]],"date-time":"2022-07-06T00:00:00Z","timestamp":1657065600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["HA 5851\/2-2"],"award-info":[{"award-number":["HA 5851\/2-2"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,7,6]]},"DOI":"10.1145\/3477495.3531743","type":"proceedings-article","created":{"date-parts":[[2022,7,7]],"date-time":"2022-07-07T15:12:13Z","timestamp":1657206733000},"page":"3131-3140","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["Axiomatic Retrieval Experimentation with ir_axioms"],"prefix":"10.1145","author":[{"given":"Alexander","family":"Bondarenko","sequence":"first","affiliation":[{"name":"Martin-Luther-Universit\u00e4t Halle-Wittenberg, Halle, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Maik","family":"Fr\u00f6be","sequence":"additional","affiliation":[{"name":"Martin-Luther-Universit\u00e4t Halle-Wittenberg, Halle, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jan Heinrich","family":"Reimer","sequence":"additional","affiliation":[{"name":"Martin-Luther-Universit\u00e4t Halle-Wittenberg, Halle, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Benno","family":"Stein","sequence":"additional","affiliation":[{"name":"Bauhaus-Universit\u00e4t Weimar, Weimar, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"V\u00f6lske","sequence":"additional","affiliation":[{"name":"Bauhaus-Universit\u00e4t Weimar, Weimar, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matthias","family":"Hagen","sequence":"additional","affiliation":[{"name":"Martin-Luther-Universit\u00e4t Halle-Wittenberg, Halle, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,7,7]]},"reference":[{"key":"e_1_3_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/1411509.1411513"},{"key":"e_1_3_2_2_2_1","unstructured":"Siddhant Arora and Andrew Yates. Investigating retrieval method selection with axiomatic features. In Proceedings of the 1st Interdisciplinary Workshop on Algorithm Selection and Meta-Learning in Information Retrieval co-located with the 41st European Conference on IR Research AMIR@ECIR 2019 (CEUR Workshop Proceedings Vol. 2360). CEUR-WS.org 18--31. http:\/\/ceur-ws.org\/Vol-2360\/paper4Axiomatic.pdf  Siddhant Arora and Andrew Yates. Investigating retrieval method selection with axiomatic features. In Proceedings of the 1st Interdisciplinary Workshop on Algorithm Selection and Meta-Learning in Information Retrieval co-located with the 41st European Conference on IR Research AMIR@ECIR 2019 (CEUR Workshop Proceedings Vol. 2360). CEUR-WS.org 18--31. http:\/\/ceur-ws.org\/Vol-2360\/paper4Axiomatic.pdf"},{"key":"e_1_3_2_2_3_1","volume-title":"Proceedings of the SIGIR 2012 Workshop on Open Source Information Retrieval, OSIR@SIGIR 2012","author":"Bialecki Andrzej","unstructured":"Andrzej Bialecki , Robert Muir , and Grant Ingersoll . Apache Lucene 4 . In Proceedings of the SIGIR 2012 Workshop on Open Source Information Retrieval, OSIR@SIGIR 2012 . University of Otago, 17--24. Andrzej Bialecki, Robert Muir, and Grant Ingersoll. Apache Lucene 4. In Proceedings of the SIGIR 2012 Workshop on Open Source Information Retrieval, OSIR@SIGIR 2012 . University of Otago, 17--24."},{"key":"e_1_3_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00051"},{"key":"e_1_3_2_2_5_1","volume-title":"Proceedings of the Twenty-Eighth Text REtrieval Conference, TREC 2019 (NIST Special Publication","volume":"2019","author":"Bondarenko Alexander","unstructured":"Alexander Bondarenko , Maik Fr\u00f6 be, Vaibhav Kasturia , Matthias Hagen , Michael V\u00f6 lske, and Benno Stein . Webis at TREC 2019: Decision track . In Proceedings of the Twenty-Eighth Text REtrieval Conference, TREC 2019 (NIST Special Publication , Vol. 1250). NIST, https:\/\/webis.de\/publications.html#bondarenko_ 2019 Alexander Bondarenko, Maik Fr\u00f6 be, Vaibhav Kasturia, Matthias Hagen, Michael V\u00f6 lske, and Benno Stein. Webis at TREC 2019: Decision track. In Proceedings of the Twenty-Eighth Text REtrieval Conference, TREC 2019 (NIST Special Publication, Vol. 1250). NIST, https:\/\/webis.de\/publications.html#bondarenko_2019"},{"key":"e_1_3_2_2_6_1","volume-title":"Proceedings of the Twenty-Seventh Text REtrieval Conference, TREC 2018 (NIST Special Publication","volume":"2018","author":"Bondarenko Alexander","unstructured":"Alexander Bondarenko , Michael V\u00f6 lske, Alexander Panchenko , Chris Biemann , Benno Stein , and Matthias Hagen . Webis at TREC 2018: Common Core track . In Proceedings of the Twenty-Seventh Text REtrieval Conference, TREC 2018 (NIST Special Publication , Vol. 500--331). NIST, https:\/\/webis.de\/publications.html#bondarenko_ 2018 Alexander Bondarenko, Michael V\u00f6 lske, Alexander Panchenko, Chris Biemann, Benno Stein, and Matthias Hagen. Webis at TREC 2018: Common Core track. In Proceedings of the Twenty-Seventh Text REtrieval Conference, TREC 2018 (NIST Special Publication, Vol. 500--331). NIST, https:\/\/webis.de\/publications.html#bondarenko_2018"},{"key":"e_1_3_2_2_7_1","first-page":"23","article-title":"From RankNet to LambdaRank to LambdaMART: An overview","volume":"11","author":"Burges Christopher J. C.","year":"2010","unstructured":"Christopher J. C. Burges . From RankNet to LambdaRank to LambdaMART: An overview . Learning , Vol. 11 , 23 -- 581 ( 2010 ), 81. Christopher J. C. Burges. From RankNet to LambdaRank to LambdaMART: An overview. Learning , Vol. 11, 23--581 (2010), 81.","journal-title":"Learning"},{"key":"e_1_3_2_2_8_1","volume-title":"Proceedings of the 42nd European Conference on IR Research, ECIR 2020 (Lecture Notes in Computer Science","volume":"618","author":"Arthur","unstructured":"Arthur C\u00e2 mara and Claudia Hauff. Diagnosing BERT with retrieval heuristics . In Proceedings of the 42nd European Conference on IR Research, ECIR 2020 (Lecture Notes in Computer Science , Vol. 12035). Springer, 605-- 618 . https:\/\/doi.org\/10.1007\/978--3-030--45439--5_40 10.1007\/978--3-030--45439--5_40 Arthur C\u00e2 mara and Claudia Hauff. Diagnosing BERT with retrieval heuristics. In Proceedings of the 42nd European Conference on IR Research, ECIR 2020 (Lecture Notes in Computer Science, Vol. 12035). Springer, 605--618. https:\/\/doi.org\/10.1007\/978--3-030--45439--5_40"},{"key":"e_1_3_2_2_9_1","volume-title":"Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019 . ACL, 195--200","author":"Chernodub Artem N.","year":"1865","unstructured":"Artem N. Chernodub , Oleksiy Oliynyk , Philipp Heidenreich , Alexander Bondarenko , Matthias Hagen , Chris Biemann , and Alexander Panchenko . textttTARGER : Neural argument mining at your fingertips . In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019 . ACL, 195--200 . https:\/\/doi.org\/10. 1865 3\/v1\/p19--3031 10.18653\/v1 Artem N. Chernodub, Oleksiy Oliynyk, Philipp Heidenreich, Alexander Bondarenko, Matthias Hagen, Chris Biemann, and Alexander Panchenko. textttTARGER: Neural argument mining at your fingertips. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019 . ACL, 195--200. https:\/\/doi.org\/10.18653\/v1\/p19--3031"},{"key":"e_1_3_2_2_10_1","volume-title":"Daniel Campos. Overview of the TREC 2020 Deep Learning track. Proceedings of the Twenty-Ninth Text REtrieval Conference, TREC 2020 (NIST Special Publication","author":"Craswell Nick","unstructured":"Nick Craswell , Bhaskar Mitra , Emine Yilmaz , and Daniel Campos. Overview of the TREC 2020 Deep Learning track. Proceedings of the Twenty-Ninth Text REtrieval Conference, TREC 2020 (NIST Special Publication , Vol. 1266). NIST. https:\/\/trec.nist.gov\/pubs\/trec29\/papers\/OVERVIEW.DL.pdf Nick Craswell, Bhaskar Mitra, Emine Yilmaz, and Daniel Campos. Overview of the TREC 2020 Deep Learning track. Proceedings of the Twenty-Ninth Text REtrieval Conference, TREC 2020 (NIST Special Publication, Vol. 1266). NIST. https:\/\/trec.nist.gov\/pubs\/trec29\/papers\/OVERVIEW.DL.pdf"},{"key":"e_1_3_2_2_11_1","volume-title":"Voorhees. Overview of the TREC 2019 Deep Learning track. Proceedings of the Twenty-Eighth Text REtrieval Conference, TREC 2019 (NIST Special Publication","author":"Craswell Nick","unstructured":"Nick Craswell , Bhaskar Mitra , Emine Yilmaz , Daniel Campos , and Ellen M . Voorhees. Overview of the TREC 2019 Deep Learning track. Proceedings of the Twenty-Eighth Text REtrieval Conference, TREC 2019 (NIST Special Publication , Vol. 1250). NIST. https:\/\/trec.nist.gov\/pubs\/trec28\/papers\/OVERVIEW.DL.pdf Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M. Voorhees. Overview of the TREC 2019 Deep Learning track. Proceedings of the Twenty-Eighth Text REtrieval Conference, TREC 2019 (NIST Special Publication, Vol. 1250). NIST. https:\/\/trec.nist.gov\/pubs\/trec28\/papers\/OVERVIEW.DL.pdf"},{"key":"e_1_3_2_2_12_1","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019","author":"Devlin Jacob","year":"1865","unstructured":"Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019 . ACL, 4171--4186. https:\/\/doi.org\/10. 1865 3\/v1\/n19--1423 10.18653\/v1 Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019. ACL, 4171--4186. https:\/\/doi.org\/10.18653\/v1\/n19--1423"},{"key":"e_1_3_2_2_13_1","volume-title":"Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2004. ACM, 49--56","author":"Fang Hui","unstructured":"Hui Fang , Tao Tao , and ChengXiang Zhai . A formal study of information retrieval heuristics . In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2004. ACM, 49--56 . https:\/\/doi.org\/10.1145\/1008992.1009004 10.1145\/1008992.1009004 Hui Fang, Tao Tao, and ChengXiang Zhai. A formal study of information retrieval heuristics. In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2004. ACM, 49--56. https:\/\/doi.org\/10.1145\/1008992.1009004"},{"key":"e_1_3_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/1961209.1961210"},{"key":"e_1_3_2_2_15_1","volume-title":"Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2006. ACM, 115--122","author":"Fang Hui","unstructured":"Hui Fang and ChengXiang Zhai . Semantic term matching in axiomatic approaches to information retrieval . In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2006. ACM, 115--122 . https:\/\/doi.org\/10.1145\/1148170.1148193 10.1145\/1148170.1148193 Hui Fang and ChengXiang Zhai. Semantic term matching in axiomatic approaches to information retrieval. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2006. ACM, 115--122. https:\/\/doi.org\/10.1145\/1148170.1148193"},{"key":"e_1_3_2_2_16_1","volume-title":"Proceedings of the 43rd European Conference on IR Research, ECIR 2021 (Lecture Notes in Computer Science","volume":"263","author":"Formal Thibault","unstructured":"Thibault Formal , Benjamin Piwowarski , and St\u00e9 phane Clinchant . A white box analysis of ColBERT . In Proceedings of the 43rd European Conference on IR Research, ECIR 2021 (Lecture Notes in Computer Science , Vol. 12657). Springer, 257-- 263 . https:\/\/doi.org\/10.1007\/978--3-030--72240--1_23 10.1007\/978--3-030--72240--1_23 Thibault Formal, Benjamin Piwowarski, and St\u00e9 phane Clinchant. A white box analysis of ColBERT. In Proceedings of the 43rd European Conference on IR Research, ECIR 2021 (Lecture Notes in Computer Science, Vol. 12657). Springer, 257--263. https:\/\/doi.org\/10.1007\/978--3-030--72240--1_23"},{"key":"e_1_3_2_2_17_1","volume-title":"Tevatron: An efficient and flexible toolkit for dense retrieval. CoRR","author":"Gao Luyu","year":"2022","unstructured":"Luyu Gao , Xueguang Ma , Jimmy Lin , and Jamie Callan . Tevatron: An efficient and flexible toolkit for dense retrieval. CoRR , Vol. abs\/ 2203 .05765 ( 2022 ). https:\/\/doi.org\/10.48550\/arXiv.2203.05765 10.48550\/arXiv.2203.05765 Luyu Gao, Xueguang Ma, Jimmy Lin, and Jamie Callan. Tevatron: An efficient and flexible toolkit for dense retrieval. CoRR , Vol. abs\/2203.05765 (2022). https:\/\/doi.org\/10.48550\/arXiv.2203.05765"},{"key":"e_1_3_2_2_18_1","volume-title":"Proceedings of the 18th International Conference on World Wide Web, WWW 2009. ACM, 381--390","author":"Gollapudi Sreenivas","unstructured":"Sreenivas Gollapudi and Aneesh Sharma . An axiomatic approach for result diversification . In Proceedings of the 18th International Conference on World Wide Web, WWW 2009. ACM, 381--390 . https:\/\/doi.org\/10.1145\/1526709.1526761 10.1145\/1526709.1526761 Sreenivas Gollapudi and Aneesh Sharma. An axiomatic approach for result diversification. In Proceedings of the 18th International Conference on World Wide Web, WWW 2009. ACM, 381--390. https:\/\/doi.org\/10.1145\/1526709.1526761"},{"key":"e_1_3_2_2_19_1","volume-title":"Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2019. ACM, 1297--1300","author":"Guo Jiafeng","unstructured":"Jiafeng Guo , Yixing Fan , Xiang Ji , and Xueqi Cheng . MatchZoo : A learning, practicing, and developing system for neural text matching . In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2019. ACM, 1297--1300 . https:\/\/doi.org\/10.1145\/3331184.3331403 10.1145\/3331184.3331403 Jiafeng Guo, Yixing Fan, Xiang Ji, and Xueqi Cheng. MatchZoo: A learning, practicing, and developing system for neural text matching. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2019. ACM, 1297--1300. https:\/\/doi.org\/10.1145\/3331184.3331403"},{"key":"e_1_3_2_2_20_1","volume-title":"Proceedings of the 25th ACM International Conference on Information and Knowledge Management, CIKM 2016 . ACM, 721--730","author":"Hagen Matthias","unstructured":"Matthias Hagen , Michael V\u00f6 lske, Steve G\u00f6 ring, and Benno Stein . Axiomatic result re-ranking . In Proceedings of the 25th ACM International Conference on Information and Knowledge Management, CIKM 2016 . ACM, 721--730 . https:\/\/doi.org\/10.1145\/2983323.2983704 10.1145\/2983323.2983704 Matthias Hagen, Michael V\u00f6 lske, Steve G\u00f6 ring, and Benno Stein. Axiomatic result re-ranking. In Proceedings of the 25th ACM International Conference on Information and Knowledge Management, CIKM 2016 . ACM, 721--730. https:\/\/doi.org\/10.1145\/2983323.2983704"},{"key":"e_1_3_2_2_21_1","first-page":"4","article-title":"a rvelin and Jaana Kek\"a l\"a inen. Cumulated gain-based evaluation of IR techniques","volume":"20","author":"Kalervo","year":"2002","unstructured":"Kalervo J \" a rvelin and Jaana Kek\"a l\"a inen. Cumulated gain-based evaluation of IR techniques . ACM Transactions on Information Systems , Vol. 20 , 4 ( 2002 ), 422--446. https:\/\/doi.org\/10.1145\/582415.582418 10.1145\/582415.582418 Kalervo J\"a rvelin and Jaana Kek\"a l\"a inen. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems , Vol. 20, 4 (2002), 422--446. https:\/\/doi.org\/10.1145\/582415.582418","journal-title":"ACM Transactions on Information Systems"},{"key":"e_1_3_2_2_22_1","volume-title":"Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020. ACM, 39--48","author":"Khattab Omar","unstructured":"Omar Khattab and Matei Zaharia . ColBERT : Efficient and effective passage search via contextualized late interaction over BERT . In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020. ACM, 39--48 . https:\/\/doi.org\/10.1145\/3397271.3401075 10.1145\/3397271.3401075 Omar Khattab and Matei Zaharia. ColBERT: Efficient and effective passage search via contextualized late interaction over BERT. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, SIGIR 2020. ACM, 39--48. https:\/\/doi.org\/10.1145\/3397271.3401075"},{"key":"e_1_3_2_2_23_1","volume-title":"Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2021. ACM, 2356--2362","author":"Lin Jimmy","unstructured":"Jimmy Lin , Xueguang Ma , Sheng-Chieh Lin , Jheng-Hong Yang , Ronak Pradeep , and Rodrigo Nogueira . Pyserini : A Python toolkit for reproducible information retrieval research with sparse and dense representations . In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2021. ACM, 2356--2362 . https:\/\/doi.org\/10.1145\/3404835.3463238 10.1145\/3404835.3463238 Jimmy Lin, Xueguang Ma, Sheng-Chieh Lin, Jheng-Hong Yang, Ronak Pradeep, and Rodrigo Nogueira. Pyserini: A Python toolkit for reproducible information retrieval research with sparse and dense representations. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2021. ACM, 2356--2362. https:\/\/doi.org\/10.1145\/3404835.3463238"},{"key":"e_1_3_2_2_24_1","volume-title":"Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2021. ACM, 2531--2535","author":"Liu Zhenghao","unstructured":"Zhenghao Liu , Kaitao Zhang , Chenyan Xiong , Zhiyuan Liu , and Maosong Sun . OpenMatch : An open source library for Neu-IR research . In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2021. ACM, 2531--2535 . https:\/\/doi.org\/10.1145\/3404835.3462789 10.1145\/3404835.3462789 Zhenghao Liu, Kaitao Zhang, Chenyan Xiong, Zhiyuan Liu, and Maosong Sun. OpenMatch: An open source library for Neu-IR research. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2021. ACM, 2531--2535. https:\/\/doi.org\/10.1145\/3404835.3462789"},{"key":"e_1_3_2_2_25_1","volume-title":"Proceedings of the 20th ACM Conference on Information and Knowledge Management, CIKM 2011 . ACM, 7--16","author":"Lv Yuanhua","year":"2063","unstructured":"Yuanhua Lv and ChengXiang Zhai . Lower-bounding term frequency normalization . In Proceedings of the 20th ACM Conference on Information and Knowledge Management, CIKM 2011 . ACM, 7--16 . https:\/\/doi.org\/10.1145\/ 2063 576.2063584 10.1145\/2063576.2063584 Yuanhua Lv and ChengXiang Zhai. Lower-bounding term frequency normalization. In Proceedings of the 20th ACM Conference on Information and Knowledge Management, CIKM 2011 . ACM, 7--16. https:\/\/doi.org\/10.1145\/2063576.2063584"},{"key":"e_1_3_2_2_26_1","volume-title":"Proceedings of the 13th ACM International Conference on Web Search and Data Mining, WSDM 2020 . ACM, 845--848","author":"MacAvaney Sean","year":"1864","unstructured":"Sean MacAvaney . OpenNIR : A complete neural ad-hoc ranking pipeline . In Proceedings of the 13th ACM International Conference on Web Search and Data Mining, WSDM 2020 . ACM, 845--848 . https:\/\/doi.org\/10.1145\/3336191.337 1864 10.1145\/3336191.3371864 Sean MacAvaney. OpenNIR: A complete neural ad-hoc ranking pipeline. In Proceedings of the 13th ACM International Conference on Web Search and Data Mining, WSDM 2020 . ACM, 845--848. https:\/\/doi.org\/10.1145\/3336191.3371864"},{"key":"e_1_3_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00457"},{"key":"e_1_3_2_2_28_1","volume-title":"Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2021. ACM, 2429--2436","author":"MacAvaney Sean","unstructured":"Sean MacAvaney , Andrew Yates , Sergey Feldman , Doug Downey , Arman Cohan , and Nazli Goharian . Simplified data wrangling with textttfontsize6.87.8selectfontir_datasets . In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2021. ACM, 2429--2436 . https:\/\/doi.org\/10.1145\/3404835.3463254 10.1145\/3404835.3463254 Sean MacAvaney, Andrew Yates, Sergey Feldman, Doug Downey, Arman Cohan, and Nazli Goharian. Simplified data wrangling with textttfontsize6.87.8selectfontir_datasets. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2021. ACM, 2429--2436. https:\/\/doi.org\/10.1145\/3404835.3463254"},{"key":"e_1_3_2_2_29_1","volume-title":"Proceedings of the SIGIR 2012 Workshop on Open Source Information Retrieval, OSIR@SIGIR 2012","author":"Macdonald Craig","unstructured":"Craig Macdonald , Richard McCreadie , Rodrygo L. T. Santos , and Iadh Ounis . From puppy to maturity: Experiences in developing Terrier . In Proceedings of the SIGIR 2012 Workshop on Open Source Information Retrieval, OSIR@SIGIR 2012 . University of Otago, 60--63. Craig Macdonald, Richard McCreadie, Rodrygo L. T. Santos, and Iadh Ounis. From puppy to maturity: Experiences in developing Terrier. In Proceedings of the SIGIR 2012 Workshop on Open Source Information Retrieval, OSIR@SIGIR 2012 . University of Otago, 60--63."},{"key":"e_1_3_2_2_30_1","volume-title":"Proceedings of the 30th ACM International Conference on Information and Knowledge Management, CIKM 2021. ACM, 4526--4533","author":"Macdonald Craig","year":"2013","unstructured":"Craig Macdonald , Nicola Tonellotto , Sean MacAvaney , and Iadh Ounis . PyTerrier : Declarative experimentation in Python from BM25 to dense retrieval . In Proceedings of the 30th ACM International Conference on Information and Knowledge Management, CIKM 2021. ACM, 4526--4533 . https:\/\/doi.org\/10.1145\/3459637.348 2013 10.1145\/3459637.3482013 Craig Macdonald, Nicola Tonellotto, Sean MacAvaney, and Iadh Ounis. PyTerrier: Declarative experimentation in Python from BM25 to dense retrieval. In Proceedings of the 30th ACM International Conference on Information and Knowledge Management, CIKM 2021. ACM, 4526--4533. https:\/\/doi.org\/10.1145\/3459637.3482013"},{"key":"e_1_3_2_2_31_1","volume-title":"Proceedings of the 9th Python in Science Conference, SciPy 2010","author":"McKinney Wes","year":"1922","unstructured":"Wes McKinney . Data structures for statistical computing in Python . In Proceedings of the 9th Python in Science Conference, SciPy 2010 . 56--61. https:\/\/doi.org\/10.25080\/Majora-92bf 1922 -00a 10.25080\/Majora-92bf1922-00a Wes McKinney. Data structures for statistical computing in Python. In Proceedings of the 9th Python in Science Conference, SciPy 2010. 56--61. https:\/\/doi.org\/10.25080\/Majora-92bf1922-00a"},{"key":"e_1_3_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/219717.219748"},{"key":"e_1_3_2_2_33_1","unstructured":"Tri Nguyen Mir Rosenberg Xia Song Jianfeng Gao Saurabh Tiwary Rangan Majumder and Li Deng. MS MARCO: A human generated MAchine Reading COmprehension dataset. In Proceedings of the Workshop on Cognitive Computation: Integrating neural and symbolic approaches 2016 co-located with the 30th Annual Conference on Neural Information Processing Systems (NIPS 2016) (CEUR Workshop Proceedings Vol. 1773). CEUR-WS.org. https:\/\/ceur-ws.org\/Vol-1773\/CoCoNIPS_2016_paper9.pdf  Tri Nguyen Mir Rosenberg Xia Song Jianfeng Gao Saurabh Tiwary Rangan Majumder and Li Deng. MS MARCO: A human generated MAchine Reading COmprehension dataset. In Proceedings of the Workshop on Cognitive Computation: Integrating neural and symbolic approaches 2016 co-located with the 30th Annual Conference on Neural Information Processing Systems (NIPS 2016) (CEUR Workshop Proceedings Vol. 1773). CEUR-WS.org. https:\/\/ceur-ws.org\/Vol-1773\/CoCoNIPS_2016_paper9.pdf"},{"key":"e_1_3_2_2_34_1","first-page":"63","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2020","author":"Nogueira Rodrigo","year":"1865","unstructured":"Rodrigo Nogueira , Zhiying Jiang , Ronak Pradeep , and Jimmy Lin . Document ranking with a pretrained sequence-to-sequence model . In Findings of the Association for Computational Linguistics: EMNLP 2020 . ACL, 708--718. https:\/\/doi.org\/10. 1865 3\/v1\/2020.findings-emnlp. 63 10.18653\/v1 Rodrigo Nogueira, Zhiying Jiang, Ronak Pradeep, and Jimmy Lin. Document ranking with a pretrained sequence-to-sequence model. In Findings of the Association for Computational Linguistics: EMNLP 2020. ACL, 708--718. https:\/\/doi.org\/10.18653\/v1\/2020.findings-emnlp.63"},{"key":"e_1_3_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.5555\/1953048.2078195"},{"key":"e_1_3_2_2_36_1","volume-title":"The expando-mono-duo design pattern for text ranking with pretrained sequence-to-sequence models. CoRR","author":"Pradeep Ronak","year":"2021","unstructured":"Ronak Pradeep , Rodrigo Nogueira , and Jimmy Lin . The expando-mono-duo design pattern for text ranking with pretrained sequence-to-sequence models. CoRR , Vol. abs\/ 2101 .05667 ( 2021 ). https:\/\/arxiv.org\/abs\/2101.05667 Ronak Pradeep, Rodrigo Nogueira, and Jimmy Lin. The expando-mono-duo design pattern for text ranking with pretrained sequence-to-sequence models. CoRR , Vol. abs\/2101.05667 (2021). https:\/\/arxiv.org\/abs\/2101.05667"},{"key":"e_1_3_2_2_37_1","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel Colin","year":"2020","unstructured":"Colin Raffel , Noam Shazeer , Adam Roberts , Katherine Lee , Sharan Narang , Michael Matena , Yanqi Zhou , Wei Li , and Peter J. Liu . Exploring the limits of transfer learning with a unified text-to-text transformer . Journal of Machine Learning Research , Vol. 21 ( 2020 ), 140: 1 -- 140 :67. https:\/\/jmlr.org\/papers\/v21\/20-074.html Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research , Vol. 21 (2020), 140:1--140:67. https:\/\/jmlr.org\/papers\/v21\/20-074.html","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_2_38_1","volume-title":"Proceedings of the 41st European Conference on IR Research, ECIR 2019 (Lecture Notes in Computer Science","volume":"503","author":"Rennings Dani\u00eb","unstructured":"Dani\u00eb l Rennings , Felipe Moraes , and Claudia Hauff . An axiomatic approach to diagnosing neural IR models . In Proceedings of the 41st European Conference on IR Research, ECIR 2019 (Lecture Notes in Computer Science , Vol. 11437). Springer, 489-- 503 . https:\/\/doi.org\/10.1007\/978--3-030--15712--8_32 10.1007\/978--3-030--15712--8_32 Dani\u00eb l Rennings, Felipe Moraes, and Claudia Hauff. An axiomatic approach to diagnosing neural IR models. In Proceedings of the 41st European Conference on IR Research, ECIR 2019 (Lecture Notes in Computer Science, Vol. 11437). Springer, 489--503. https:\/\/doi.org\/10.1007\/978--3-030--15712--8_32"},{"key":"e_1_3_2_2_39_1","volume-title":"Proceedings of The Third Text REtrieval Conference, TREC 1994 (NIST Special Publication","volume":"126","author":"Robertson Stephen E.","unstructured":"Stephen E. Robertson , Steve Walker , Susan Jones , Micheline Hancock-Beaulieu , and Mike Gatford . Okapi at TREC-3 . In Proceedings of The Third Text REtrieval Conference, TREC 1994 (NIST Special Publication , Vol. 500--225). NIST, 109-- 126 . https:\/\/trec.nist.gov\/pubs\/trec3\/papers\/city.ps.gz Stephen E. Robertson, Steve Walker, Susan Jones, Micheline Hancock-Beaulieu, and Mike Gatford. Okapi at TREC-3. In Proceedings of The Third Text REtrieval Conference, TREC 1994 (NIST Special Publication, Vol. 500--225). NIST, 109--126. https:\/\/trec.nist.gov\/pubs\/trec3\/papers\/city.ps.gz"},{"key":"e_1_3_2_2_40_1","volume-title":"Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2019. ACM, 981--984","author":"Rosset Corby","unstructured":"Corby Rosset , Bhaskar Mitra , Chenyan Xiong , Nick Craswell , Xia Song , and Saurabh Tiwary . An axiomatic approach to regularizing neural ranking models . In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2019. ACM, 981--984 . https:\/\/doi.org\/10.1145\/3331184.3331296 10.1145\/3331184.3331296 Corby Rosset, Bhaskar Mitra, Chenyan Xiong, Nick Craswell, Xia Song, and Saurabh Tiwary. An axiomatic approach to regularizing neural ranking models. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2019. ACM, 981--984. https:\/\/doi.org\/10.1145\/3331184.3331296"},{"key":"e_1_3_2_2_41_1","volume-title":"Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2005. ACM, 488--495","author":"Shi Shuming","unstructured":"Shuming Shi , Ji-Rong Wen , Qing Yu , Ruihua Song , and Wei-Ying Ma . Gravitation-based model for information retrieval . In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2005. ACM, 488--495 . https:\/\/doi.org\/10.1145\/1076034.1076117 10.1145\/1076034.1076117 Shuming Shi, Ji-Rong Wen, Qing Yu, Ruihua Song, and Wei-Ying Ma. Gravitation-based model for information retrieval. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2005. ACM, 488--495. https:\/\/doi.org\/10.1145\/1076034.1076117"},{"key":"e_1_3_2_2_42_1","volume-title":"Proceedings of the 2021 ACM SIGIR International Conference on the Theory of Information Retrieval, ICTIR 2021. ACM, 13--22","author":"Michael V\u00f6","unstructured":"Michael V\u00f6 lske, Alexander Bondarenko , Maik Fr\u00f6 be, Benno Stein , Jaspreet Singh , Matthias Hagen , and Avishek Anand . Towards axiomatic explanations for neural ranking models . In Proceedings of the 2021 ACM SIGIR International Conference on the Theory of Information Retrieval, ICTIR 2021. ACM, 13--22 . https:\/\/doi.org\/10.1145\/3471158.3472256 10.1145\/3471158.3472256 Michael V\u00f6 lske, Alexander Bondarenko, Maik Fr\u00f6 be, Benno Stein, Jaspreet Singh, Matthias Hagen, and Avishek Anand. Towards axiomatic explanations for neural ranking models. In Proceedings of the 2021 ACM SIGIR International Conference on the Theory of Information Retrieval, ICTIR 2021. ACM, 13--22. https:\/\/doi.org\/10.1145\/3471158.3472256"},{"key":"e_1_3_2_2_43_1","volume-title":"Proceedings of the 34th European Conference on IR Research, ECIR 2012 (Lecture Notes in Computer Science","volume":"120","author":"Wu Hao","unstructured":"Hao Wu and Hui Fang . Relation based term weighting regularization . In Proceedings of the 34th European Conference on IR Research, ECIR 2012 (Lecture Notes in Computer Science , Vol. 7224). Springer, 109-- 120 . https:\/\/doi.org\/10.1007\/978--3--642--28997--2_10 10.1007\/978--3--642--28997--2_10 Hao Wu and Hui Fang. Relation based term weighting regularization. In Proceedings of the 34th European Conference on IR Research, ECIR 2012 (Lecture Notes in Computer Science, Vol. 7224). Springer, 109--120. https:\/\/doi.org\/10.1007\/978--3--642--28997--2_10"},{"key":"e_1_3_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-009-9112-1"},{"key":"e_1_3_2_2_45_1","volume-title":"Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2017. ACM, 1253--1256","author":"Yang Peilin","unstructured":"Peilin Yang , Hui Fang , and Jimmy Lin . Anserini : Enabling the use of Lucene for information retrieval research . In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2017. ACM, 1253--1256 . https:\/\/doi.org\/10.1145\/3077136.3080721 10.1145\/3077136.3080721 Peilin Yang, Hui Fang, and Jimmy Lin. Anserini: Enabling the use of Lucene for information retrieval research. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2017. ACM, 1253--1256. https:\/\/doi.org\/10.1145\/3077136.3080721"},{"key":"e_1_3_2_2_46_1","first-page":"4","article-title":"Reproducible ranking baselines using Lucene","volume":"10","author":"Yang Peilin","year":"2018","unstructured":"Peilin Yang , Hui Fang , and Jimmy Lin . Anserini : Reproducible ranking baselines using Lucene . ACM Journal of Data and Information Quality , Vol. 10 , 4 ( 2018 ), 16:1--16:20. https:\/\/doi.org\/10.1145\/3239571 10.1145\/3239571 Peilin Yang, Hui Fang, and Jimmy Lin. Anserini: Reproducible ranking baselines using Lucene. ACM Journal of Data and Information Quality , Vol. 10, 4 (2018), 16:1--16:20. https:\/\/doi.org\/10.1145\/3239571","journal-title":"ACM Journal of Data and Information Quality"},{"key":"e_1_3_2_2_47_1","volume-title":"Proceedings of the 29th ACM International Conference on Information and Knowledge Management, CIKM 2020 . ACM, 3181--3188","author":"Yates Andrew","unstructured":"Andrew Yates , Kevin Martin Jose , Xinyu Zhang , and Jimmy Lin . Flexible IR pipelines with Capreolus . In Proceedings of the 29th ACM International Conference on Information and Knowledge Management, CIKM 2020 . ACM, 3181--3188 . https:\/\/doi.org\/10.1145\/3340531.3412780 10.1145\/3340531.3412780 Andrew Yates, Kevin Martin Jose, Xinyu Zhang, and Jimmy Lin. Flexible IR pipelines with Capreolus. In Proceedings of the 29th ACM International Conference on Information and Knowledge Management, CIKM 2020 . ACM, 3181--3188. https:\/\/doi.org\/10.1145\/3340531.3412780"},{"key":"e_1_3_2_2_48_1","volume-title":"Proceedings of the 32nd European Conference on IR Research, ECIR 2010 (Lecture Notes in Computer Science","volume":"356","author":"Zheng Wei","unstructured":"Wei Zheng and Hui Fang . Query aspect based term weighting regularization in information retrieval . In Proceedings of the 32nd European Conference on IR Research, ECIR 2010 (Lecture Notes in Computer Science , Vol. 5993). Springer, 344-- 356 . https:\/\/doi.org\/10.1007\/978--3--642--12275-0_31 10.1007\/978--3--642--12275-0_31 Wei Zheng and Hui Fang. Query aspect based term weighting regularization in information retrieval. In Proceedings of the 32nd European Conference on IR Research, ECIR 2010 (Lecture Notes in Computer Science, Vol. 5993). Springer, 344--356. https:\/\/doi.org\/10.1007\/978--3--642--12275-0_31"}],"event":{"name":"SIGIR '22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval","location":"Madrid Spain","acronym":"SIGIR '22","sponsor":["SIGIR ACM Special Interest Group on Information Retrieval"]},"container-title":["Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3477495.3531743","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3477495.3531743","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:02:07Z","timestamp":1750186927000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3477495.3531743"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,6]]},"references-count":48,"alternative-id":["10.1145\/3477495.3531743","10.1145\/3477495"],"URL":"https:\/\/doi.org\/10.1145\/3477495.3531743","relation":{},"subject":[],"published":{"date-parts":[[2022,7,6]]},"assertion":[{"value":"2022-07-07","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}