{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T22:31:45Z","timestamp":1759962705675,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":54,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,8,23]],"date-time":"2022-08-23T00:00:00Z","timestamp":1661212800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,8,23]]},"DOI":"10.1145\/3539813.3545153","type":"proceedings-article","created":{"date-parts":[[2022,8,25]],"date-time":"2022-08-25T22:18:32Z","timestamp":1661465912000},"page":"13-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["On the Effect of Ranking Axioms on IR Evaluation Metrics"],"prefix":"10.1145","author":[{"given":"Fernando","family":"Giner","sequence":"first","affiliation":[{"name":"E.T.S.I. Inform\u00e1tica UNED, Madrid, Spain"}]}],"member":"320","published-online":{"date-parts":[[2022,8,25]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3077136.3084369"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-020-09376-y"},{"key":"e_1_3_2_1_3_1","volume-title":"Proceedings of the SIGIR'17 Workshop on Axiomatic Thinking for Information Retrieval and Related Tasks (ATIR).","author":"Amig\u00f3 Enrique","year":"2017","unstructured":"Enrique Amig\u00f3 , Fernando Giner , Julio Gonzalo , and Felisa Verdejo . 2017 . An axiomatic account of similarity . In Proceedings of the SIGIR'17 Workshop on Axiomatic Thinking for Information Retrieval and Related Tasks (ATIR). Enrique Amig\u00f3, Fernando Giner, Julio Gonzalo, and Felisa Verdejo. 2017. An axiomatic account of similarity. In Proceedings of the SIGIR'17 Workshop on Axiomatic Thinking for Information Retrieval and Related Tasks (ATIR)."},{"key":"e_1_3_2_1_4_1","volume-title":"A comparison of extrinsic clustering evaluation metrics based on formal constraints. Information retrieval 12, 4","author":"Amig\u00f3 Enrique","year":"2009","unstructured":"Enrique Amig\u00f3 , Julio Gonzalo , Javier Artiles , and Felisa Verdejo . 2009. A comparison of extrinsic clustering evaluation metrics based on formal constraints. Information retrieval 12, 4 ( 2009 ), 461--486. Enrique Amig\u00f3, Julio Gonzalo, Javier Artiles, and Felisa Verdejo. 2009. A comparison of extrinsic clustering evaluation metrics based on formal constraints. Information retrieval 12, 4 (2009), 461--486."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2484028.2484081"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-020-09374-0"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3210024"},{"volume-title":"Lattice theory","author":"Birkhoff Garrett","key":"e_1_3_2_1_8_1","unstructured":"Garrett Birkhoff . 1940. Lattice theory . Vol. 25 . American Mathematical Soc . Garrett Birkhoff. 1940. Lattice theory. Vol. 25. American Mathematical Soc."},{"key":"e_1_3_2_1_9_1","volume-title":"Information Retrieval","author":"Blair David C.","year":"1979","unstructured":"David C. Blair . 1979. Information Retrieval , 2 nd ed. C.J. Van Rijsbergen. London : Butterworths; 1979 : 208 pp. Price: $32.50. JASIS 30, 6 (1979), 374--375. https: \/\/doi.org\/10.1002\/asi.4630300621 David C. Blair. 1979. Information Retrieval, 2nd ed. C.J. Van Rijsbergen. London: Butterworths; 1979: 208 pp. Price: $32.50. JASIS 30, 6 (1979), 374--375. https: \/\/doi.org\/10.1002\/asi.4630300621","edition":"2"},{"key":"e_1_3_2_1_10_1","volume-title":"Distributive lattices. Lattices and Ordered Algebraic Structures","author":"Blyth TS","year":"2005","unstructured":"TS Blyth . 2005. Distributive lattices. Lattices and Ordered Algebraic Structures ( 2005 ), 65--76. TS Blyth. 2005. Distributive lattices. Lattices and Ordered Algebraic Structures (2005), 65--76."},{"key":"e_1_3_2_1_11_1","volume-title":"SIGIR","volume":"84","author":"Bollmann Peter","year":"1984","unstructured":"Peter Bollmann . 1984 . Two axioms for evaluation measures in information retrieval . In SIGIR , Vol. 84 . Citeseer, 233--245. Peter Bollmann. 1984. Two axioms for evaluation measures in information retrieval. In SIGIR, Vol. 84. Citeseer, 233--245."},{"volume-title":"Proceedings of the 3rd annual ACM conference on Research and development in information retrieval. Citeseer, 256--267","author":"Bollmann Peter","key":"e_1_3_2_1_12_1","unstructured":"Peter Bollmann and Vladimir S. Cherniavsky . 1980. Measurement-theoretical investigation of the MZ-metric . In Proceedings of the 3rd annual ACM conference on Research and development in information retrieval. Citeseer, 256--267 . Peter Bollmann and Vladimir S. Cherniavsky. 1980. Measurement-theoretical investigation of the MZ-metric. In Proceedings of the 3rd annual ACM conference on Research and development in information retrieval. Citeseer, 256--267."},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2499178.2499182"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2009916.2010037"},{"volume-title":"Aslib proceedings","author":"Cleverdon Cyril","key":"e_1_3_2_1_15_1","unstructured":"Cyril Cleverdon . 1967. The Cranfield tests on index language devices . In Aslib proceedings , Vol. 19 . MCB UP Ltd , 173--194. Cyril Cleverdon. 1967. The Cranfield tests on index language devices. In Aslib proceedings, Vol. 19. MCB UP Ltd, 173--194."},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-23318-0_10"},{"key":"e_1_3_2_1_17_1","volume-title":"Proceedings of the 2013 Conference on the Theory of Information Retrieval. 6--13","author":"Clinchant St\u00e9phane","year":"2013","unstructured":"St\u00e9phane Clinchant and Eric Gaussier . 2013 . A theoretical analysis of pseudorelevance feedback models . In Proceedings of the 2013 Conference on the Theory of Information Retrieval. 6--13 . St\u00e9phane Clinchant and Eric Gaussier. 2013. A theoretical analysis of pseudorelevance feedback models. In Proceedings of the 2013 Conference on the Theory of Information Retrieval. 6--13."},{"key":"e_1_3_2_1_18_1","volume-title":"An information-theoretic external cluster-validity measure. arXiv preprint arXiv:1301.0565","author":"Dom Byron E","year":"2012","unstructured":"Byron E Dom . 2012. An information-theoretic external cluster-validity measure. arXiv preprint arXiv:1301.0565 ( 2012 ). Byron E Dom. 2012. An information-theoretic external cluster-validity measure. arXiv preprint arXiv:1301.0565 (2012)."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1008992.1009004"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1961209.1961210"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1076034.1076116"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148193"},{"key":"e_1_3_2_1_24_1","volume-title":"Towards Meaningful Statements in IR Evaluation. Mapping Evaluation Measures to Interval Scales. arXiv preprint arXiv:2101.02668","author":"Ferrante Marco","year":"2021","unstructured":"Marco Ferrante , Nicola Ferro , and Norbert Fuhr . 2021. Towards Meaningful Statements in IR Evaluation. Mapping Evaluation Measures to Interval Scales. arXiv preprint arXiv:2101.02668 ( 2021 ). Marco Ferrante, Nicola Ferro, and Norbert Fuhr. 2021. Towards Meaningful Statements in IR Evaluation. Mapping Evaluation Measures to Interval Scales. arXiv preprint arXiv:2101.02668 (2021)."},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2808194.2809452"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2018.2840708"},{"key":"e_1_3_2_1_27_1","volume-title":"ACM SIGIR Forum","volume":"54","author":"Fuhr Norbert","year":"2021","unstructured":"Norbert Fuhr . 2021 . Proof by experimentation? towards better IR research . In ACM SIGIR Forum , Vol. 54 . ACM New York, NY, USA, 1--4. Norbert Fuhr. 2021. Proof by experimentation? towards better IR research. In ACM SIGIR Forum, Vol. 54. ACM New York, NY, USA, 1--4."},{"volume-title":"General lattice theory","author":"Gr\u00e4tzer George","key":"e_1_3_2_1_28_1","unstructured":"George Gr\u00e4tzer . 2002. General lattice theory . Springer Science & Business Media . George Gr\u00e4tzer. 2002. General lattice theory. Springer Science & Business Media."},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2983323.2983704"},{"key":"e_1_3_2_1_30_1","unstructured":"Theodorus Wilhelmus Charles Huibers. 1996. An axiomatic theory for information retrieval. Ph. D. Dissertation.  Theodorus Wilhelmus Charles Huibers. 1996. An axiomatic theory for information retrieval. Ph. D. Dissertation."},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-28997-2_23"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.10137"},{"key":"e_1_3_2_1_33_1","volume-title":"Additive and polynomial representations.","author":"Krantz David","year":"1971","unstructured":"David Krantz , Duncan Luce , Patrick Suppes , and Amos Tversky . 1971. Foundations of measurement , Vol. I : Additive and polynomial representations. ( 1971 ). David Krantz, Duncan Luce, Patrick Suppes, and Amos Tversky. 1971. Foundations of measurement, Vol. I: Additive and polynomial representations. (1971)."},{"key":"e_1_3_2_1_34_1","volume-title":"Foundations of Measurement","author":"Krantz David H","year":"1989","unstructured":"David H Krantz . 1989. Foundations of Measurement . Vol. II . Geometrical, Threshold and Probabilistic Representations . ( 1989 ). David H Krantz. 1989. Foundations of Measurement. Vol. II. Geometrical, Threshold and Probabilistic Representations. (1989)."},{"key":"e_1_3_2_1_35_1","volume-title":"Axiometrics: Axioms of Information Retrieval Effectiveness Metrics.. In EVIA@ NTCIR.","author":"Maddalena Eddy","year":"2014","unstructured":"Eddy Maddalena and Stefano Mizzaro . 2014 . Axiometrics: Axioms of Information Retrieval Effectiveness Metrics.. In EVIA@ NTCIR. Eddy Maddalena and Stefano Mizzaro. 2014. Axiometrics: Axioms of Information Retrieval Effectiveness Metrics.. In EVIA@ NTCIR."},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jmva.2006.11.013"},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-45068-6_1"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2505515.2507665"},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/1416950.1416952"},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2911451.2914768"},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-020-09372-2"},{"key":"e_1_3_2_1_42_1","volume-title":"Measurement theory. Encyclopedia of Mathematics and its applications 7","author":"Roberts Fred S","year":"1985","unstructured":"Fred S Roberts . 1985. Measurement theory. Encyclopedia of Mathematics and its applications 7 ( 1985 ). Fred S Roberts. 1985. Measurement theory. Encyclopedia of Mathematics and its applications 7 (1985)."},{"key":"e_1_3_2_1_43_1","volume-title":"Proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning (EMNLP-CoNLL). 410--420","author":"Rosenberg Andrew","year":"2007","unstructured":"Andrew Rosenberg and Julia Hirschberg . 2007 . V-measure: A conditional entropybased external cluster evaluation measure . In Proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning (EMNLP-CoNLL). 410--420 . Andrew Rosenberg and Julia Hirschberg. 2007. V-measure: A conditional entropybased external cluster evaluation measure. In Proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning (EMNLP-CoNLL). 410--420."},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331296"},{"volume-title":"Studies in Pure Mathematics (Presented to Richard Rado)","author":"Rota Gian-Carlo","key":"e_1_3_2_1_45_1","unstructured":"Gian-Carlo Rota . 1971. On the combinatorics of the Euler characteristic . In Studies in Pure Mathematics (Presented to Richard Rado) . Academic Press London , 221--233. Gian-Carlo Rota. 1971. On the combinatorics of the Euler characteristic. In Studies in Pure Mathematics (Presented to Richard Rado). Academic Press London, 221--233."},{"key":"e_1_3_2_1_46_1","volume-title":"ACM SIGIR Forum","volume":"54","author":"Sakai Tetsuya","year":"2021","unstructured":"Tetsuya Sakai . 2021 . On Fuhr's guideline for IR evaluation . In ACM SIGIR Forum , Vol. 54 . ACM New York, NY, USA, 1--8. Tetsuya Sakai. 2021. On Fuhr's guideline for IR evaluation. In ACM SIGIR Forum, Vol. 54. ACM New York, NY, USA, 1--8."},{"key":"e_1_3_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-008-9059-7"},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2808194.2809449"},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-019-09363-y"},{"key":"e_1_3_2_1_50_1","volume-title":"Proceedings of the Workshop on Testing of Deployable Learning and Decision Systems, the 19th Neural Information Processing Systems Conference (NIPS","author":"Sokolova Marina","year":"2006","unstructured":"Marina Sokolova . 2006 . Assessing invariance properties of evaluation measures . In Proceedings of the Workshop on Testing of Deployable Learning and Decision Systems, the 19th Neural Information Processing Systems Conference (NIPS 2006). Marina Sokolova. 2006. Assessing invariance properties of evaluation measures. In Proceedings of the Workshop on Testing of Deployable Learning and Decision Systems, the 19th Neural Information Processing Systems Conference (NIPS 2006)."},{"key":"e_1_3_2_1_51_1","volume-title":"Information retrieval systems. Science 141, 3577","author":"Swets John A","year":"1963","unstructured":"John A Swets . 1963. Information retrieval systems. Science 141, 3577 ( 1963 ), 245--250. John A Swets. 1963. Information retrieval systems. Science 141, 3577 (1963), 245--250."},{"key":"e_1_3_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1108\/eb026584"},{"key":"e_1_3_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3471158.3472256"},{"key":"e_1_3_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.5555\/208098.208105"},{"key":"e_1_3_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/2499178.2499205"}],"event":{"name":"ICTIR '22: The 2022 ACM SIGIR International Conference on the Theory of Information Retrieval","sponsor":["SIGIR ACM Special Interest Group on Information Retrieval"],"location":"Madrid Spain","acronym":"ICTIR '22"},"container-title":["Proceedings of the 2022 ACM SIGIR International Conference on Theory of Information Retrieval"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3539813.3545153","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3539813.3545153","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T18:10:04Z","timestamp":1750183804000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3539813.3545153"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,23]]},"references-count":54,"alternative-id":["10.1145\/3539813.3545153","10.1145\/3539813"],"URL":"https:\/\/doi.org\/10.1145\/3539813.3545153","relation":{},"subject":[],"published":{"date-parts":[[2022,8,23]]},"assertion":[{"value":"2022-08-25","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}