{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,20]],"date-time":"2026-02-20T03:08:47Z","timestamp":1771556927037,"version":"3.50.1"},"reference-count":19,"publisher":"MIT Press - Journals","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["TACL"],"published-print":{"date-parts":[[2014,12]]},"abstract":"<jats:p> Annotated data is prerequisite for many NLP applications. Acquiring large-scale annotated corpora is a major bottleneck, requiring significant time and resources. Recent work has proposed turning annotation into a game to increase its appeal and lower its cost; however, current games are largely text-based and closely resemble traditional annotation tasks. We propose a new linguistic annotation paradigm that produces annotations from playing graphical video games. The effectiveness of this design is demonstrated using two video games: one to create a mapping from WordNet senses to images, and a second game that performs Word Sense Disambiguation. Both games produce accurate results. The first game yields annotation quality equal to that of experts and a cost reduction of 73% over equivalent crowdsourcing; the second game provides a 16.3% improvement in accuracy over current state-of-the-art sense disambiguation games with WordNet. <\/jats:p>","DOI":"10.1162\/tacl_a_00195","type":"journal-article","created":{"date-parts":[[2018,12,28]],"date-time":"2018-12-28T15:43:26Z","timestamp":1546011806000},"page":"449-464","source":"Crossref","is-referenced-by-count":22,"title":["It\u2019s All Fun and Games until Someone Annotates: Video Games with a                     Purpose for Linguistic Annotation"],"prefix":"10.1162","volume":"2","author":[{"given":"David","family":"Jurgens","sequence":"first","affiliation":[{"name":"Department of Computer Science, Sapienza University of Rome,"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Roberto","family":"Navigli","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Sapienza University of Rome,"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","reference":[{"key":"p_2","doi-asserted-by":"publisher","DOI":"10.1162\/coli.07-034-R2"},{"key":"p_4","first-page":"1107","volume":"3","author":"Barnard Kobus","year":"2003","journal-title":"The Journal of Machine Learning Research"},{"issue":"7307","key":"p_9","doi-asserted-by":"crossref","first-page":"756","DOI":"10.1038\/nature09304","volume":"466","author":"Cooper Seth","year":"2010","journal-title":"Nature"},{"issue":"4","key":"p_11","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1017\/S1351324902002966","volume":"8","author":"Edmonds Philip","year":"2002","journal-title":"Natural Language Engineering"},{"issue":"3","key":"p_13","first-page":"511","volume":"39","author":"Erk Katrin","year":"2012","journal-title":"Computational Linguistics"},{"issue":"4","key":"p_16","doi-asserted-by":"crossref","first-page":"797","DOI":"10.1109\/TPAMI.2012.118","volume":"35","author":"Feng Yansong","year":"2013","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"issue":"4","key":"p_18","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2337542.2337544","volume":"3","author":"Herda\u011fdelen Ama\u00e7","year":"2012","journal-title":"ACM Transactions on Intelligent Systems and Technology"},{"key":"p_21","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1016\/j.artint.2012.10.002","volume":"194","author":"Hovy Eduard H.","year":"2013","journal-title":"Artificial Intelligence"},{"issue":"1","key":"p_23","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1023\/A:1002407003264","volume":"34","author":"Krishnamurthy Ramesh","year":"2000","journal-title":"Computers and the Humanities"},{"issue":"12","key":"p_24","doi-asserted-by":"crossref","first-page":"2891","DOI":"10.1109\/TPAMI.2012.162","volume":"35","author":"Kulkarni Girish","year":"2013","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"issue":"2","key":"p_28","doi-asserted-by":"crossref","first-page":"476","DOI":"10.1109\/TPAMI.2012.100","volume":"35","author":"Mensink Thomas","year":"2013","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"p_30","doi-asserted-by":"crossref","first-page":"217","DOI":"10.1016\/j.artint.2012.07.001","volume":"193","author":"Navigli Roberto","year":"2012","journal-title":"Artificial Intelligence"},{"issue":"2","key":"p_31","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1459352.1459355","volume":"41","author":"Navigli Roberto","year":"2009","journal-title":"ACM Computing Surveys (CSUR)"},{"issue":"2","key":"p_32","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1017\/S135132490500402X","volume":"13","author":"Palmer Martha","year":"2007","journal-title":"Natural Language Engineering"},{"issue":"2","key":"p_34","first-page":"209","volume":"46","author":"Passonneau Rebecca J.","year":"2012","journal-title":"Language Resources and Evaluation"},{"issue":"1","key":"p_37","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2448116.2448119","volume":"3","author":"Poesio Massimo","year":"2013","journal-title":"ACM Transactions on Interactive Intelligent Systems"},{"issue":"11","key":"p_40","doi-asserted-by":"crossref","first-page":"1958","DOI":"10.1109\/TPAMI.2008.128","volume":"30","author":"Torralba Antonio","year":"2008","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"issue":"8","key":"p_44","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1145\/1378704.1378719","volume":"51","author":"von Ahn Luis","year":"2008","journal-title":"Communications of the ACM"},{"issue":"1","key":"p_45","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1007\/s10579-012-9176-1","volume":"47","author":"Wang Aobo","year":"2013","journal-title":"Language Resources and Evaluation"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/tacl_a_00195","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:38:59Z","timestamp":1615585139000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/43338"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,12]]},"references-count":19,"alternative-id":["10.1162\/tacl_a_00195"],"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00195","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2014,12]]}}}