{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,4]],"date-time":"2025-11-04T23:36:58Z","timestamp":1762299418286,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":38,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,7,25]],"date-time":"2020-07-25T00:00:00Z","timestamp":1595635200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Australian Research Council","award":["DP190101113"],"award-info":[{"award-number":["DP190101113"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,7,25]]},"DOI":"10.1145\/3397271.3401033","type":"proceedings-article","created":{"date-parts":[[2020,7,25]],"date-time":"2020-07-25T07:50:08Z","timestamp":1595663408000},"page":"339-348","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Bayesian Inferential Risk Evaluation On Multiple IR Systems"],"prefix":"10.1145","author":[{"given":"Rodger","family":"Benham","sequence":"first","affiliation":[{"name":"RMIT University, Melbourne, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ben","family":"Carterette","sequence":"additional","affiliation":[{"name":"Spotify, New York, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"J. Shane","family":"Culpepper","sequence":"additional","affiliation":[{"name":"RMIT University, Melbourne, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alistair","family":"Moffat","sequence":"additional","affiliation":[{"name":"The University of Melbourne, Melbourne, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2020,7,25]]},"reference":[{"key":"e_1_3_2_2_1_1","first-page":"1","volume-title":"TREC 2017 common core track overview. In Proc. TREC","author":"Allan J.","year":"2017","unstructured":"J. Allan , D. Harman , E. Kanoulas , D. Li , C. Van Gysel , and E. M. Voorhees . TREC 2017 common core track overview. In Proc. TREC , pages 1 -- 14 , 2017 . J. Allan, D. Harman, E. Kanoulas, D. Li, C. Van Gysel, and E. M. Voorhees. TREC 2017 common core track overview. In Proc. TREC, pages 1--14, 2017."},{"key":"e_1_3_2_2_2_1","first-page":"1147","volume-title":"SIGIR 2015 workshop on reproducibility, inexplicability, and generalizability of results (RIGOR). In Proc. SIGIR","author":"Arguello J.","year":"2015","unstructured":"J. Arguello , F. Diaz , J. Lin , and A. Trotman . SIGIR 2015 workshop on reproducibility, inexplicability, and generalizability of results (RIGOR). In Proc. SIGIR , pages 1147 -- 1148 , 2015 . J. Arguello, F. Diaz, J. Lin, and A. Trotman. SIGIR 2015 workshop on reproducibility, inexplicability, and generalizability of results (RIGOR). In Proc. SIGIR, pages 1147--1148, 2015."},{"key":"e_1_3_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1645953.1646031"},{"issue":"2","key":"e_1_3_2_2_4_1","first-page":"171","article-title":"A class of distributions which includes the normal ones","volume":"12","author":"Azzalini A.","year":"1985","unstructured":"A. Azzalini . A class of distributions which includes the normal ones . Scand. J. Stat. , 12 ( 2 ): 171 -- 178 , 1985 . A. Azzalini. A class of distributions which includes the normal ones. Scand. J. Stat., 12 (2): 171--178, 1985.","journal-title":"Scand. J. Stat."},{"key":"e_1_3_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jml.2012.11.001"},{"key":"e_1_3_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3166072.3166084"},{"key":"e_1_3_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3372124.3372125"},{"key":"e_1_3_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-23318-0_11"},{"key":"e_1_3_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2808194.2809469"},{"key":"e_1_3_2_2_10_1","volume-title":"Factors determining the performance of indexing systems. Cranfield: College of Aeronautics, 1 (28)","author":"Cleverdon C. W.","year":"1966","unstructured":"C. W. Cleverdon , J. Mills , and M. Keen . Factors determining the performance of indexing systems. Cranfield: College of Aeronautics, 1 (28) , 1966 . C. W. Cleverdon, J. Mills, and M. Keen. Factors determining the performance of indexing systems. Cranfield: College of Aeronautics, 1 (28), 1966."},{"key":"e_1_3_2_2_11_1","volume-title":"TREC 2013 web track overview. In Proc. TREC","author":"Collins-Thompson K.","year":"2014","unstructured":"K. Collins-Thompson , C. Macdonald , P. Bennett , F. Diaz , and E. M. Voorhees . TREC 2013 web track overview. In Proc. TREC , 2014 . K. Collins-Thompson, C. Macdonald, P. Bennett, F. Diaz, and E. M. Voorhees. TREC 2013 web track overview. In Proc. TREC, 2014."},{"key":"e_1_3_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/1571941.1572114"},{"key":"e_1_3_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2911451.2911511"},{"key":"e_1_3_2_2_14_1","first-page":"26","volume-title":"Proc. ECIR","author":"B.","year":"2014","unstructured":"B. Dincc er, I. Ounis , and C. Macdonald . Tackling biased baselines in the risk-sensitive evaluation of retrieval systems . In Proc. ECIR , pages 26 -- 38 , 2014 a. B. Dincc er, I. Ounis, and C. Macdonald. Tackling biased baselines in the risk-sensitive evaluation of retrieval systems. In Proc. ECIR, pages 26--38, 2014 a."},{"key":"e_1_3_2_2_15_1","first-page":"23","volume-title":"Proc. SIGIR","author":"B. T.","year":"2014","unstructured":"B. T. Dincc er, C. Macdonald , and I. Ounis . Hypothesis testing for the risk-sensitive evaluation of retrieval systems . In Proc. SIGIR , pages 23 -- 32 , 2014 b. B. T. Dincc er, C. Macdonald, and I. Ounis. Hypothesis testing for the risk-sensitive evaluation of retrieval systems. In Proc. SIGIR, pages 23--32, 2014 b."},{"key":"e_1_3_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2911451.2911530"},{"key":"e_1_3_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3310364"},{"key":"e_1_3_2_2_18_1","first-page":"243","volume-title":"Proc. TREC","author":"Fox E. A.","year":"1994","unstructured":"E. A. Fox and J. A. Shaw . Combination of multiple searches . Proc. TREC , pages 243 -- 252 , 1994 . E. A. Fox and J. A. Shaw. Combination of multiple searches. Proc. TREC, pages 243--252, 1994."},{"key":"e_1_3_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3190580.3190586"},{"key":"e_1_3_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1214\/ss\/1177011136"},{"issue":"2","key":"e_1_3_2_2_21_1","first-page":"189","article-title":"Why we (usually) don't have to worry about multiple comparisons","volume":"5","author":"Gelman A.","year":"2012","unstructured":"A. Gelman , J. Hill , and M. Yajima . Why we (usually) don't have to worry about multiple comparisons . J. Res. Int. Educ. , 5 ( 2 ): 189 -- 211 , 2012 . A. Gelman, J. Hill, and M. Yajima. Why we (usually) don't have to worry about multiple comparisons. J. Res. Int. Educ., 5 (2): 189--211, 2012.","journal-title":"J. Res. Int. Educ."},{"key":"e_1_3_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1201\/b16018"},{"issue":"1","key":"e_1_3_2_2_23_1","first-page":"1593","article-title":"The No-U-Turn sampler: Adaptively setting path lengths","volume":"15","author":"Hoffman M. D.","year":"2014","unstructured":"M. D. Hoffman and A. Gelman . The No-U-Turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo. J. Mach. Learn. Res , 15 ( 1 ): 1593 -- 1623 , 2014 . M. D. Hoffman and A. Gelman. The No-U-Turn sampler: Adaptively setting path lengths in Hamiltonian Monte Carlo. J. Mach. Learn. Res, 15 (1): 1593--1623, 2014.","journal-title":"Hamiltonian Monte Carlo. J. Mach. Learn. Res"},{"key":"e_1_3_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1561\/1500000051"},{"key":"e_1_3_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/582415.582418"},{"key":"e_1_3_2_2_26_1","volume-title":"JAGS, and Stan","author":"Kruschke J.","year":"2014","unstructured":"J. Kruschke . Doing Bayesian data analysis: A tutorial with R , JAGS, and Stan . Academic Press , 2014 . J. Kruschke. Doing Bayesian data analysis: A tutorial with R, JAGS, and Stan. Academic Press, 2014."},{"key":"e_1_3_2_2_27_1","volume-title":"A student's guide to Bayesian statistics","author":"Lambert B.","year":"2018","unstructured":"B. Lambert . A student's guide to Bayesian statistics . Sage , 2018 . B. Lambert. A student's guide to Bayesian statistics. Sage, 2018."},{"issue":"1","key":"e_1_3_2_2_28_1","first-page":"2","article-title":"Rank-biased precision for measurement of retrieval effectiveness","volume":"27","author":"Moffat A.","unstructured":"A. Moffat and J. Zobel . Rank-biased precision for measurement of retrieval effectiveness . ACM Trans. Inf. Sys. , 27 ( 1 ): 2 .1--2.27, 2008. A. Moffat and J. Zobel. Rank-biased precision for measurement of retrieval effectiveness. ACM Trans. Inf. Sys., 27 (1): 2.1--2.27, 2008.","journal-title":"ACM Trans. Inf. Sys."},{"key":"e_1_3_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.20982\/tqmp.14.2.p099"},{"key":"e_1_3_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.24203"},{"key":"e_1_3_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2911451.2911492"},{"key":"e_1_3_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3077136.3080766"},{"key":"e_1_3_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-008-9059-7"},{"key":"e_1_3_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/1321440.1321528"},{"key":"e_1_3_2_2_35_1","volume-title":"T. C. Rosa, and W. S. Martins. Risk-sensitive learning to rank with evolutionary multi-objective feature selection. ACM Trans. Inf. Sys., 37 (2): 1--34","author":"Sousa D. X.","year":"2019","unstructured":"lves, Rosa, and Martins]sousa2019risk D. X. Sousa , S. Canuto , M. A. Goncc alves , T. C. Rosa, and W. S. Martins. Risk-sensitive learning to rank with evolutionary multi-objective feature selection. ACM Trans. Inf. Sys., 37 (2): 1--34 , 2019 . lves, Rosa, and Martins]sousa2019riskD. X. Sousa, S. Canuto, M. A. Goncc alves, T. C. Rosa, and W. S. Martins. Risk-sensitive learning to rank with evolutionary multi-objective feature selection. ACM Trans. Inf. Sys., 37 (2): 1--34, 2019."},{"key":"e_1_3_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331259"},{"key":"e_1_3_2_2_37_1","first-page":"69","volume-title":"Overview of TREC 2004 robust retrieval track. In Proc. TREC","author":"Voorhees E. M.","year":"2004","unstructured":"E. M. Voorhees . Overview of TREC 2004 robust retrieval track. In Proc. TREC , pages 69 -- 77 , 2004 . E. M. Voorhees. Overview of TREC 2004 robust retrieval track. In Proc. TREC, pages 69--77, 2004."},{"key":"e_1_3_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2348283.2348385"}],"event":{"name":"SIGIR '20: The 43rd International ACM SIGIR conference on research and development in Information Retrieval","sponsor":["SIGIR ACM Special Interest Group on Information Retrieval"],"location":"Virtual Event China","acronym":"SIGIR '20"},"container-title":["Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3397271.3401033","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3397271.3401033","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:31:37Z","timestamp":1750195897000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3397271.3401033"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,25]]},"references-count":38,"alternative-id":["10.1145\/3397271.3401033","10.1145\/3397271"],"URL":"https:\/\/doi.org\/10.1145\/3397271.3401033","relation":{},"subject":[],"published":{"date-parts":[[2020,7,25]]},"assertion":[{"value":"2020-07-25","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}