{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T15:09:30Z","timestamp":1772809770471,"version":"3.50.1"},"reference-count":26,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2014,12,23]],"date-time":"2014-12-23T00:00:00Z","timestamp":1419292800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGIR Forum"],"published-print":{"date-parts":[[2014,12,23]]},"abstract":"<jats:p>In this paper we overview the 2014 Entity Recognition and Disambiguation Challenge (ERD'14), which took place from March to June 2014 and was summarized in a dedicated workshop at SIGIR 2014. The main goal of the ERD challenge was to promote research in recognition and disambiguation of entities in unstructured text. Unlike many past entity linking challenges, no mention segmentations were given to the participating systems for a given document. Participants were asked to implement a web service for their system to minimize human involvement during evaluation and to enable measuring the processing times. The challenge has attracted a lot of interest (over 100 teams registered, and 27 of those submitted final results).<\/jats:p>\n          <jats:p>In this paper we cover the task definition, issues encountered during annotation, and provide a detailed analysis of all the participating systems. Specifically, we show how we adapted the pooling technique to address the difficulties of gathering annotations for the entity linking task. We also summarize the ERD workshop that followed the challenge, including the oral and poster presentations as well as the invited talks.<\/jats:p>","DOI":"10.1145\/2701583.2701591","type":"journal-article","created":{"date-parts":[[2015,1,5]],"date-time":"2015-01-05T13:27:09Z","timestamp":1420464429000},"page":"63-77","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":35,"title":["ERD'14"],"prefix":"10.1145","volume":"48","author":[{"given":"David","family":"Carmel","sequence":"first","affiliation":[{"name":"Yahoo! Labs"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ming-Wei","family":"Chang","sequence":"additional","affiliation":[{"name":"Microsoft Research"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Evgeniy","family":"Gabrilovich","sequence":"additional","affiliation":[{"name":"Google"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bo-June (Paul)","family":"Hsu","sequence":"additional","affiliation":[{"name":"Microsoft Research"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kuansan","family":"Wang","sequence":"additional","affiliation":[{"name":"Microsoft Research"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2014,12,23]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2633211.2634357"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2633211.2634363"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2633211.2634348"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2633211.2634360"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2506182.2506198"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2633211.2634347"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2633211.2634349"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1871437.1871689"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2633211.2634354"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2633211.2634356"},{"key":"e_1_2_1_11_1","volume-title":"Proceedings of the TAC 2011 Workshop","author":"Ji H.","year":"2011","unstructured":"H. Ji , R. Grishman , and Dang. Overview of the tac 2011 knowledge base population track . In Proceedings of the TAC 2011 Workshop , 2011 . H. Ji, R. Grishman, and Dang. Overview of the tac 2011 knowledge base population track. In Proceedings of the TAC 2011 Workshop, 2011."},{"key":"e_1_2_1_12_1","volume-title":"Third Text Analysis Conference (TAC 2010)","author":"Ji Heng","year":"2010","unstructured":"Heng Ji , Ralph Grishman , Hoa Trang Dang , Kira Griffitt , and Joe Ellis . Overview of the tac 2010 knowledge base population track . In Third Text Analysis Conference (TAC 2010) , 2010 . Heng Ji, Ralph Grishman, Hoa Trang Dang, Kira Griffitt, and Joe Ellis. Overview of the tac 2010 knowledge base population track. In Third Text Analysis Conference (TAC 2010), 2010."},{"key":"e_1_2_1_13_1","volume-title":"Report on the need for and provision of an \"ideal\" information retrieval test collection","author":"SPARCK","year":"1975","unstructured":"SPARCK JONES. Report on the need for and provision of an \"ideal\" information retrieval test collection . 1975 . SPARCK JONES. Report on the need for and provision of an \"ideal\" information retrieval test collection. 1975."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2633211.2634358"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2633211.2639488"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2633211.2634352"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2633211.2634351"},{"key":"e_1_2_1_18_1","first-page":"111","volume-title":"Text Analysis Conference (TAC 2009","volume":"17","author":"McNamee Paul","year":"2009","unstructured":"Paul McNamee and Hoa Trang Dang . Overview of the tac 2009 knowledge base population track . In Text Analysis Conference (TAC 2009 ), volume 17 , pages 111 -- 113 , 2009 . Paul McNamee and Hoa Trang Dang. Overview of the tac 2009 knowledge base population track. In Text Analysis Conference (TAC 2009), volume 17, pages 111--113, 2009."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2633211.2634359"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2633211.2639489"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2633211.2634353"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2633211.2634350"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2633211.2634361"},{"key":"e_1_2_1_24_1","volume-title":"Overview of trec","author":"Voorhees Ellen M","year":"2001","unstructured":"Ellen M Voorhees and Donna Harman . Overview of trec 2001 . In Trec , 2001. Ellen M Voorhees and Donna Harman. Overview of trec 2001. In Trec, 2001."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2633211.2634362"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/290941.291014"}],"container-title":["ACM SIGIR Forum"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2701583.2701591","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2701583.2701591","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T06:13:11Z","timestamp":1750227191000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2701583.2701591"}},"subtitle":["Entity Recognition and Disambiguation Challenge"],"short-title":[],"issued":{"date-parts":[[2014,12,23]]},"references-count":26,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2014,12,23]]}},"alternative-id":["10.1145\/2701583.2701591"],"URL":"https:\/\/doi.org\/10.1145\/2701583.2701591","relation":{},"ISSN":["0163-5840"],"issn-type":[{"value":"0163-5840","type":"print"}],"subject":[],"published":{"date-parts":[[2014,12,23]]},"assertion":[{"value":"2014-12-23","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}