{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T00:50:54Z","timestamp":1774399854963,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":22,"publisher":"ACM","license":[{"start":{"date-parts":[[2018,6,27]],"date-time":"2018-06-27T00:00:00Z","timestamp":1530057600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"European Union's Horizon 2020 research and innovation programme","award":["732328"],"award-info":[{"award-number":["732328"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2018,6,27]]},"DOI":"10.1145\/3209978.3210052","type":"proceedings-article","created":{"date-parts":[[2018,7,2]],"date-time":"2018-07-02T12:12:40Z","timestamp":1530533560000},"page":"675-684","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":21,"title":["On Fine-Grained Relevance Scales"],"prefix":"10.1145","author":[{"given":"Kevin","family":"Roitero","sequence":"first","affiliation":[{"name":"University of Udine, Udine, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Eddy","family":"Maddalena","sequence":"additional","affiliation":[{"name":"University of Southampton, Southampton, United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gianluca","family":"Demartini","sequence":"additional","affiliation":[{"name":"University of Queensland, Brisbane, Australia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Stefano","family":"Mizzaro","sequence":"additional","affiliation":[{"name":"University of Udine, Udine, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2018,6,27]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2012.01.004"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1645953.1646033"},{"key":"e_1_3_2_1_3_1","volume-title":"Overview of the TREC 2004 Terabyte Track.. In TREC","volume":"4","author":"Clarke Charles LA","year":"2004","unstructured":"Charles LA Clarke , Nick Craswell , and Ian Soboroff . 2004 . Overview of the TREC 2004 Terabyte Track.. In TREC , Vol. Vol. 4 . 74. Charles LA Clarke, Nick Craswell, and Ian Soboroff . 2004. Overview of the TREC 2004 Terabyte Track.. In TREC, Vol. Vol. 4. 74."},{"key":"e_1_3_2_1_4_1","volume-title":"TREC 2014 web track overview. Technical Report. MICHIGAN UNIV ANN ARBOR.","author":"Collins-Thompson Kevyn","year":"2015","unstructured":"Kevyn Collins-Thompson , Craig Macdonald , Paul Bennett , Fernando Diaz , and Ellen M Voorhees . 2015 . TREC 2014 web track overview. Technical Report. MICHIGAN UNIV ANN ARBOR. Kevyn Collins-Thompson, Craig Macdonald, Paul Bennett, Fernando Diaz, and Ellen M Voorhees . 2015. TREC 2014 web track overview. Technical Report. MICHIGAN UNIV ANN ARBOR."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3159652.3159654"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130914"},{"key":"e_1_3_2_1_7_1","volume-title":"Psychophysics: the fundamentals","author":"Gescheider George A","unstructured":"George A Gescheider . 2013. Psychophysics: the fundamentals . Psychology Press . George A Gescheider . 2013. Psychophysics: the fundamentals. Psychology Press."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-28997-2_16"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/TBC.2010.2086750"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/582415.582418"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/582415.582418"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3077136.3080840"},{"key":"e_1_3_2_1_13_1","volume-title":"Computing Krippendorff's alpha reliability. Departmental papers (ASC)","author":"Klaus Krippendorff","year":"2007","unstructured":"Klaus Krippendorff . 2007. Computing Krippendorff's alpha reliability. Departmental papers (ASC) ( 2007 ), 43. Klaus Krippendorff . 2007. Computing Krippendorff's alpha reliability. Departmental papers (ASC) (2007), 43."},{"key":"e_1_3_2_1_14_1","volume-title":"Dante Degl'Innocenti, Stefano Mizzaro, and Gianluca Demartini .","author":"Maddalena Eddy","year":"2016","unstructured":"Eddy Maddalena , Marco Basaldella , Dario De Nart , Dante Degl'Innocenti, Stefano Mizzaro, and Gianluca Demartini . 2016 . Crowdsourcing relevance assessments: The unexpected benefits of limiting the time to judge Fourth AAAI Conference on Human Computation and Crowdsourcing . Eddy Maddalena, Marco Basaldella, Dario De Nart, Dante Degl'Innocenti, Stefano Mizzaro, and Gianluca Demartini . 2016. Crowdsourcing relevance assessments: The unexpected benefits of limiting the time to judge Fourth AAAI Conference on Human Computation and Crowdsourcing."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3002172"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"crossref","unstructured":"Tyler McDonnell Matthew Lease Mucahid Kutlu and Tamer Elsayed . 2016. Why is that relevant? Collecting annotator rationales for relevance judgments Fourth AAAI Conference on Human Computation and Crowdsourcing.  Tyler McDonnell Matthew Lease Mucahid Kutlu and Tamer Elsayed . 2016. Why is that relevant? Collecting annotator rationales for relevance judgments Fourth AAAI Conference on Human Computation and Crowdsourcing.","DOI":"10.24963\/ijcai.2017\/692"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2006.07.020"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.v58:13"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/564376.564433"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(1999)50:3%3C254::AID-ASI8%3E3.3.CO;2-P"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766462.2767760"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2566486.2567989"}],"event":{"name":"SIGIR '18: The 41st International ACM SIGIR conference on research and development in Information Retrieval","location":"Ann Arbor MI USA","acronym":"SIGIR '18","sponsor":["SIGIR ACM Special Interest Group on Information Retrieval"]},"container-title":["The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3209978.3210052","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3209978.3210052","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T02:07:49Z","timestamp":1750212469000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3209978.3210052"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,6,27]]},"references-count":22,"alternative-id":["10.1145\/3209978.3210052","10.1145\/3209978"],"URL":"https:\/\/doi.org\/10.1145\/3209978.3210052","relation":{},"subject":[],"published":{"date-parts":[[2018,6,27]]},"assertion":[{"value":"2018-06-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}