{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:24:28Z","timestamp":1750220668828,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":34,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,10,19]],"date-time":"2020-10-19T00:00:00Z","timestamp":1603065600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,10,19]]},"DOI":"10.1145\/3340531.3412878","type":"proceedings-article","created":{"date-parts":[[2020,10,19]],"date-time":"2020-10-19T05:31:06Z","timestamp":1603085466000},"page":"3031-3038","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Fine-Grained Relevance Annotations for Multi-Task Document Ranking and Question Answering"],"prefix":"10.1145","author":[{"given":"Sebastian","family":"Hofst\u00e4tter","sequence":"first","affiliation":[{"name":"TU Wien, Vienna, Austria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Markus","family":"Zlabinger","sequence":"additional","affiliation":[{"name":"TU Wien, Vienna, Austria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mete","family":"Sertkan","sequence":"additional","affiliation":[{"name":"TU Wien, Vienna, Austria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael","family":"Schr\u00f6der","sequence":"additional","affiliation":[{"name":"TU Wien, Vienna, Austria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Allan","family":"Hanbury","sequence":"additional","affiliation":[{"name":"TU Wien, Vienna, Austria"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2020,10,19]]},"reference":[{"key":"e_1_3_2_2_1_1","volume-title":"Proc. of EMNLP.","author":"Yilmaz Zeynep Akkalyoncu","year":"2019","unstructured":"Zeynep Akkalyoncu Yilmaz , Wei Yang , Haotian Zhang , and Jimmy Lin . 2019 . Cross-Domain Modeling of Sentence-Level Evidence for Document Retrieval . In Proc. of EMNLP. Zeynep Akkalyoncu Yilmaz, Wei Yang, Haotian Zhang, and Jimmy Lin. 2019. Cross-Domain Modeling of Sentence-Level Evidence for Document Retrieval. In Proc. of EMNLP."},{"key":"e_1_3_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2012.01.004"},{"key":"e_1_3_2_2_3_1","volume-title":"Proc. of NIPS.","author":"Bajaj Payal","year":"2016","unstructured":"Payal Bajaj , Daniel Campos , Nick Craswell , 2016 . MS MARCO : A Human Generated MAchine Reading COmprehension Dataset . In Proc. of NIPS. Payal Bajaj, Daniel Campos, Nick Craswell, et al. 2016. MS MARCO : A Human Generated MAchine Reading COmprehension Dataset. In Proc. of NIPS."},{"key":"e_1_3_2_2_4_1","volume-title":"The concept of relevance in IR. Journal of the American Society for IST","author":"Borlund Pia","year":"2003","unstructured":"Pia Borlund . 2003. The concept of relevance in IR. Journal of the American Society for IST ( 2003 ). Pia Borlund. 2003. The concept of relevance in IR. Journal of the American Society for IST (2003)."},{"key":"e_1_3_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331373"},{"key":"e_1_3_2_2_6_1","volume-title":"Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457","author":"Clark Peter","year":"2018","unstructured":"Peter Clark , Isaac Cowhey , Oren Etzioni , Tushar Khot , Ashish Sabharwal , Carissa Schoenick , and Oyvind Tafjord . 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457 ( 2018 ). Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457 (2018)."},{"key":"e_1_3_2_2_7_1","volume-title":"Overview of the TREC 2019 deep learning track. In In Proc. of TREC.","author":"Craswell Nick","year":"2019","unstructured":"Nick Craswell , Bhaskar Mitra , Emine Yilmaz , and Daniel Campos . 2019 . Overview of the TREC 2019 deep learning track. In In Proc. of TREC. Nick Craswell, Bhaskar Mitra, Emine Yilmaz, and Daniel Campos. 2019. Overview of the TREC 2019 deep learning track. In In Proc. of TREC."},{"key":"e_1_3_2_2_8_1","volume-title":"Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805","author":"Devlin Jacob","year":"2018","unstructured":"Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . 2018 . Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018). Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)."},{"key":"e_1_3_2_2_9_1","volume-title":"Searchqa: A new q&a dataset augmented with context from a search engine. arXiv preprint arXiv:1704.05179","author":"Dunn Matthew","year":"2017","unstructured":"Matthew Dunn , Levent Sagun , Mike Higgins , V Ugur Guney , Volkan Cirik , and Kyunghyun Cho . 2017 . Searchqa: A new q&a dataset augmented with context from a search engine. arXiv preprint arXiv:1704.05179 (2017). Matthew Dunn, Levent Sagun, Mike Higgins, V Ugur Guney, Volkan Cirik, and Kyunghyun Cho. 2017. Searchqa: A new q&a dataset augmented with context from a search engine. arXiv preprint arXiv:1704.05179 (2017)."},{"key":"e_1_3_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2499149.2499168"},{"key":"e_1_3_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-45442-5_21"},{"key":"e_1_3_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401224"},{"key":"e_1_3_2_2_13_1","volume-title":"Proc. of TREC.","author":"Hofst\u00e4tter Sebastian","year":"2019","unstructured":"Sebastian Hofst\u00e4tter , Markus Zlabinger , and Allan Hanbury . 2019 . TU Wien @ TREC Deep Learning '19 -- Simple Contextualization for Re-ranking . In Proc. of TREC. Sebastian Hofst\u00e4tter, Markus Zlabinger, and Allan Hanbury. 2019. TU Wien @ TREC Deep Learning '19 -- Simple Contextualization for Re-ranking. In Proc. of TREC."},{"key":"e_1_3_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-45442-5_58"},{"key":"e_1_3_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3269206.3271779"},{"key":"e_1_3_2_2_16_1","doi-asserted-by":"crossref","unstructured":"Sarthak Jain Madeleine van Zuylen Hannaneh Hajishirzi and Iz Beltagy. 2020. SciREX: A Challenge Dataset for Document-Level Information Extraction. arXiv:cs.CL\/2005.00512  Sarthak Jain Madeleine van Zuylen Hannaneh Hajishirzi and Iz Beltagy. 2020. SciREX: A Challenge Dataset for Document-Level Information Extraction. arXiv:cs.CL\/2005.00512","DOI":"10.18653\/v1\/2020.acl-main.670"},{"key":"e_1_3_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313707"},{"key":"e_1_3_2_2_18_1","volume-title":"Overview of WebCLEF","author":"Jijkoun Valentin","year":"2007","unstructured":"Valentin Jijkoun and Maarten de Rijke . 2008. Overview of WebCLEF 2007 , Carol Peters, Valentin Jijkoun , Thomas Mandl, Henning M\u00fcller, Douglas W. Oard, Anselmo Pe\u00f1as, Vivien Petras, and Diana Santos (Eds .). Valentin Jijkoun and Maarten de Rijke. 2008. Overview of WebCLEF 2007, Carol Peters, Valentin Jijkoun, Thomas Mandl, Henning M\u00fcller, Douglas W. Oard, Anselmo Pe\u00f1as, Vivien Petras, and Diana Santos (Eds.)."},{"key":"e_1_3_2_2_19_1","unstructured":"Yohan Jo Elijah Mayfield Chris Reed and Eduard Hovy. 2020. Machine-Aided Annotation for Fine-Grained Proposition Types in Argumentation.  Yohan Jo Elijah Mayfield Chris Reed and Eduard Hovy. 2020. Machine-Aided Annotation for Fine-Grained Proposition Types in Argumentation."},{"key":"e_1_3_2_2_20_1","doi-asserted-by":"crossref","unstructured":"Tom Kwiatkowski Jennimaria Palomaki Olivia Redfield Michael Collins Ankur Parikh Chris Alberti Danielle Epstein etal 2019. Natural Questions: a Benchmark for Question Answering Research. Trans. of the ACL (2019).  Tom Kwiatkowski Jennimaria Palomaki Olivia Redfield Michael Collins Ankur Parikh Chris Alberti Danielle Epstein et al. 2019. Natural Questions: a Benchmark for Question Answering Research. Trans. of the ACL (2019).","DOI":"10.1162\/tacl_a_00276"},{"key":"e_1_3_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1260"},{"key":"e_1_3_2_2_22_1","volume-title":"Learning to attend on essential terms: An enhanced retriever-reader model for open-domain question answering. arXiv preprint arXiv:1808.09492","author":"Ni Jianmo","year":"2018","unstructured":"Jianmo Ni , Chenguang Zhu , Weizhu Chen , and Julian McAuley . 2018. Learning to attend on essential terms: An enhanced retriever-reader model for open-domain question answering. arXiv preprint arXiv:1808.09492 ( 2018 ). Jianmo Ni, Chenguang Zhu, Weizhu Chen, and Julian McAuley. 2018. Learning to attend on essential terms: An enhanced retriever-reader model for open-domain question answering. arXiv preprint arXiv:1808.09492 (2018)."},{"key":"e_1_3_2_2_23_1","volume-title":"Answering while summarizing: Multi-task learning for multi-hop qa with evidence extraction. arXiv preprint arXiv:1905.08511","author":"Nishida Kosuke","year":"2019","unstructured":"Kosuke Nishida , Kyosuke Nishida , Masaaki Nagata , Atsushi Otsuka , Itsumi Saito , Hisako Asano , and Junji Tomita . 2019. Answering while summarizing: Multi-task learning for multi-hop qa with evidence extraction. arXiv preprint arXiv:1905.08511 ( 2019 ). Kosuke Nishida, Kyosuke Nishida, Masaaki Nagata, Atsushi Otsuka, Itsumi Saito, Hisako Asano, and Junji Tomita. 2019. Answering while summarizing: Multi-task learning for multi-hop qa with evidence extraction. arXiv preprint arXiv:1905.08511 (2019)."},{"key":"e_1_3_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3269206.3271702"},{"key":"e_1_3_2_2_25_1","volume-title":"Proc. of CLEF.","author":"Overwijk Arnold","year":"2008","unstructured":"Arnold Overwijk , Dong Nguyen , Claudia Hauff , Rudolf Berend Trieschnigg , Djoerd Hiemstra , and Franciska MG de Jong . 2008 . On the Evaluation of Snippet Selection for Information Retrieval . In Proc. of CLEF. Arnold Overwijk, Dong Nguyen, Claudia Hauff, Rudolf Berend Trieschnigg, Djoerd Hiemstra, and Franciska MG de Jong. 2008. On the Evaluation of Snippet Selection for Information Retrieval. In Proc. of CLEF."},{"key":"e_1_3_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-44564-9_4"},{"key":"e_1_3_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1264"},{"key":"e_1_3_2_2_28_1","doi-asserted-by":"crossref","unstructured":"Kirk Roberts Sonya E Shooshan Laritza Rodriguez Swapna Abhyankar Halil Kilicoglu and Dina Demner-Fushman. 2015. The role of fine-grained annotations in supervised recognition of risk factors for heart disease from EHRs. Journal of Bio.Inf. (2015).  Kirk Roberts Sonya E Shooshan Laritza Rodriguez Swapna Abhyankar Halil Kilicoglu and Dina Demner-Fushman. 2015. The role of fine-grained annotations in supervised recognition of risk factors for heart disease from EHRs. Journal of Bio.Inf. (2015).","DOI":"10.1016\/j.jbi.2015.06.010"},{"key":"e_1_3_2_2_29_1","volume-title":"Real-time open-domain question answering with dense-sparse phrase index. arXiv preprint arXiv:1906.05807","author":"Seo Minjoon","year":"2019","unstructured":"Minjoon Seo , Jinhyuk Lee , Tom Kwiatkowski , Ankur P Parikh , Ali Farhadi , and Hannaneh Hajishirzi . 2019. Real-time open-domain question answering with dense-sparse phrase index. arXiv preprint arXiv:1906.05807 ( 2019 ). Minjoon Seo, Jinhyuk Lee, Tom Kwiatkowski, Ankur P Parikh, Ali Farhadi, and Hannaneh Hajishirzi. 2019. Real-time open-domain question answering with dense-sparse phrase index. arXiv preprint arXiv:1906.05807 (2019)."},{"key":"e_1_3_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3366423.3380305"},{"key":"e_1_3_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331233"},{"key":"e_1_3_2_2_32_1","volume-title":"Proc. of TREC.","author":"Yan Ming","year":"2019","unstructured":"Ming Yan , Chenliang Li , Chen Wu , Bin Bi , Wei Wang , Jiangnan Xia , and Luo Si . 2019 . IDST at TREC 2019 Deep Learning Track . In Proc. of TREC. Ming Yan, Chenliang Li, Chen Wu, Bin Bi, Wei Wang, Jiangnan Xia, and Luo Si. 2019. IDST at TREC 2019 Deep Learning Track. In Proc. of TREC."},{"key":"e_1_3_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1074"},{"key":"e_1_3_2_2_34_1","volume-title":"RLTM: an efficient neural IR framework for long documents. arXiv preprint arXiv:1906.09404","author":"Zheng Chen","year":"2019","unstructured":"Chen Zheng , Yu Sun , Shengxian Wan , and Dianhai Yu. 2019. RLTM: an efficient neural IR framework for long documents. arXiv preprint arXiv:1906.09404 ( 2019 ). Chen Zheng, Yu Sun, Shengxian Wan, and Dianhai Yu. 2019. RLTM: an efficient neural IR framework for long documents. arXiv preprint arXiv:1906.09404 (2019)."}],"event":{"name":"CIKM '20: The 29th ACM International Conference on Information and Knowledge Management","sponsor":["SIGWEB ACM Special Interest Group on Hypertext, Hypermedia, and Web","SIGIR ACM Special Interest Group on Information Retrieval"],"location":"Virtual Event Ireland","acronym":"CIKM '20"},"container-title":["Proceedings of the 29th ACM International Conference on Information & Knowledge Management"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3340531.3412878","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3340531.3412878","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:02:56Z","timestamp":1750197776000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3340531.3412878"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,10,19]]},"references-count":34,"alternative-id":["10.1145\/3340531.3412878","10.1145\/3340531"],"URL":"https:\/\/doi.org\/10.1145\/3340531.3412878","relation":{},"subject":[],"published":{"date-parts":[[2020,10,19]]},"assertion":[{"value":"2020-10-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}