{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,21]],"date-time":"2025-10-21T00:38:42Z","timestamp":1761007122227,"version":"build-2065373602"},"reference-count":31,"publisher":"Wiley","issue":"1","license":[{"start":{"date-parts":[[2013,1,24]],"date-time":"2013-01-24T00:00:00Z","timestamp":1358985600000},"content-version":"vor","delay-in-days":389,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc of Assoc for Info"],"published-print":{"date-parts":[[2012,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Crowdsourcing and Games with a Purpose (GWAP) have each received considerable attention in recent years. These two human computation mechanisms assist with tasks that cannot be solved by computers alone. Despite this increased attention, much of this transformation has been limited to a few aspects of Information Retrieval (IR). In this paper, we examine these two mechanisms' applicability to IR. Using an IR model, we apply criteria to determine the suitability of these crowdsourcing and GWAP mechanisms to each step of the model. Our analysis illustrates that these mechanisms can apply to several of these steps with good returns.<\/jats:p>","DOI":"10.1002\/meet.14504901050","type":"journal-article","created":{"date-parts":[[2013,1,24]],"date-time":"2013-01-24T10:49:23Z","timestamp":1359024563000},"page":"1-10","source":"Crossref","is-referenced-by-count":2,"title":["Applying human computation mechanisms to information retrieval"],"prefix":"10.1002","volume":"49","author":[{"given":"Christopher G.","family":"Harris","sequence":"first","affiliation":[]},{"given":"Padmini","family":"Srinivasan","sequence":"additional","affiliation":[]}],"member":"311","published-online":{"date-parts":[[2013,1,24]]},"reference":[{"key":"e_1_2_5_2_1","unstructured":"Alonso O.andMizzaro S.(2009)Can we get rid of TREC assessors? Using Mechanical Turk for relevance assessment. In Proc SIGIR'09 ACM New York NY."},{"key":"e_1_2_5_3_1","doi-asserted-by":"crossref","unstructured":"Bennett P. N. Chickering D. M.et al. (2009)Learning consensus opinion: mining data from a labeling game. In Proc. WWW '09. ACM New York NY pp 121\u2013130.","DOI":"10.1145\/1526709.1526727"},{"key":"e_1_2_5_4_1","doi-asserted-by":"crossref","unstructured":"Bernstein M. S. Little G.et. al. (2010)Soylent: a word processor with a crowd inside. In Proc UIST'10. ACM New York NY pp 313\u2013322.","DOI":"10.1145\/1866029.1866078"},{"volume-title":"Flow: The psychology of optimal experience","year":"1991","author":"Csikszentmihalyi M.","key":"e_1_2_5_5_1"},{"key":"e_1_2_5_6_1","doi-asserted-by":"crossref","unstructured":"Eickhoff C. Harris C. G. de Vries A.P. andP.Srinivasan. (2012)Quality through flow and immersion: gamifying crowdsourced relevance assessments. In Proc. SIGIR 2012 ACM New York NY.","DOI":"10.1145\/2348283.2348400"},{"key":"e_1_2_5_7_1","unstructured":"Finin T. W.Murnane et al. (2010).Annotating named entities in Twitter data with crowdsourcing. In Proc. NAACL HLT 2010 Association for Computational Linguistics Stroudsburg PA USA. pp 80\u201388."},{"key":"e_1_2_5_8_1","doi-asserted-by":"crossref","unstructured":"Franklin M. D.Kossmann et al. (2011).CrowdDB: Answering queries with crowdsourcing. In Proc. SIGMOD (2011). ACM New York USA. pp 61\u201372.","DOI":"10.1145\/1989323.1989331"},{"volume-title":"Crowdsourcing document relevance assessment with Mechanical Turk","year":"2010","author":"Grady C.","key":"e_1_2_5_9_1"},{"key":"e_1_2_5_10_1","doi-asserted-by":"crossref","unstructured":"Hacker S.andL.von Ahn(2009).Matchin: eliciting user preferences with an online game In Proc CHI'09 ACM. New York USA pp 1207\u20131216.","DOI":"10.1145\/1518701.1518882"},{"key":"e_1_2_5_11_1","doi-asserted-by":"crossref","unstructured":"Kumar A.andM.Lease(2011).Learning to rank from a noisy crowd. In Proc SIGIR'11 ACM New York pp 1221\u20131222.","DOI":"10.1145\/2009916.2010129"},{"volume-title":"Information Retrieval Today","year":"1993","author":"Lancaster F. W.","key":"e_1_2_5_12_1"},{"key":"e_1_2_5_13_1","unstructured":"Lieberman H. D.Smith et al. (2007).Common Consensus: a web\u2010based game for collecting commonsense goals. In Proc. IUI'07 ACM New York NY."},{"key":"e_1_2_5_14_1","doi-asserted-by":"crossref","unstructured":"Little G. Chilton L. B. et. al. (2010).TurKit: human computation algorithms on mechanical turk. InProc. UIST '10. ACM New York NY pp.57\u201366.","DOI":"10.1145\/1866029.1866040"},{"key":"e_1_2_5_15_1","unstructured":"Luon Y. C.Aperjis et al.Rankr: A Mobile System for Crowdsourcing Opinions."},{"key":"e_1_2_5_16_1","doi-asserted-by":"crossref","unstructured":"Ma H. R.Chandrasekar et al. (2009).Improving search engines using human computation games In Proc. HCOMP'09 ACM New York NY.","DOI":"10.1145\/1645953.1645990"},{"volume-title":"Building data integration systems: A mass collaboration approach","year":"2003","author":"McCann R.","key":"e_1_2_5_17_1"},{"key":"e_1_2_5_18_1","unstructured":"McGonigal J.(2011).Reality is broken: Why games make us better and how they can change the world Penguin Pr."},{"key":"e_1_2_5_19_1","doi-asserted-by":"crossref","unstructured":"Milne D. D. M.Nichols et al. (2008).A competitive environment for exploratory query expansion In JCDL'08 ACM New York NY.","DOI":"10.1145\/1378889.1378922"},{"key":"e_1_2_5_20_1","doi-asserted-by":"crossref","unstructured":"Naveed N. T.Gottron et al. (2011).Searching Microblogs: Coping with Sparsity and Document Quality. In Proc. CIKM'11. ACM New York NY pp 183\u2013188.","DOI":"10.1145\/2063576.2063607"},{"key":"e_1_2_5_21_1","unstructured":"Ng V.(2007).Semantic class induction and coreference resolution. In Proc. ACL'07 ACL Stroudsburg PA USA pp.536\u2013543."},{"key":"e_1_2_5_22_1","doi-asserted-by":"crossref","unstructured":"Nowak S.andS.R\u00fcger(2010).Reliable Annotations via Crowdsourcing. In Proc. MIR'10. ACM New York NY. pp.557\u2013566.","DOI":"10.1145\/1743384.1743478"},{"key":"e_1_2_5_23_1","unstructured":"Paiement J. F. J. G.Shanahan et al. (2010).Crowd Sourcing Local Search Relevance. In Proc. CrowdConf 2010. ACM New York NY."},{"key":"e_1_2_5_24_1","doi-asserted-by":"crossref","unstructured":"Ravi S.andK.Knight(2009).Minimized models for unsupervised part\u2010of\u2010speech tagging In Proc. ACL'09 ACL Stroudsburg PA USA. pp 504\u2013512.","DOI":"10.3115\/1687878.1687950"},{"key":"e_1_2_5_25_1","doi-asserted-by":"crossref","unstructured":"Robson C. S.Kandel et al. (2011).Data collection by the people for the people. In Proc. CHI'11. Vancouver BC Canada ACM New York pp 25\u201328.","DOI":"10.1145\/1979742.1979569"},{"key":"e_1_2_5_26_1","doi-asserted-by":"crossref","unstructured":"Su Q. D.Pavlov et al. (2007).Internet\u2010scale collection of human\u2010reviewed data. In Proc. WWW '07. Banff Alberta Canada ACM New York. NY pp 231\u2013240.","DOI":"10.1145\/1242572.1242604"},{"key":"e_1_2_5_27_1","unstructured":"Urbano J. J.Morato et al. (2010).Crowdsourcing preference judgments for evaluation of music similarity tasks. In Proc. CSE'10 ACM New York NY. pp 9\u201316."},{"key":"e_1_2_5_28_1","doi-asserted-by":"crossref","unstructured":"von Ahn L.andL.Dabbish(2004).Labeling images with a computer game In Proc. CHI'04 ACM New York NY.","DOI":"10.1145\/985692.985733"},{"key":"e_1_2_5_29_1","doi-asserted-by":"crossref","unstructured":"von Ahn L. M.Kedia et al. (2006).Verbosity: a game for collecting common\u2010sense facts In Proc. CHI'06 ACM New York NY pp 75\u201378.","DOI":"10.1145\/1124772.1124784"},{"key":"e_1_2_5_30_1","first-page":"2035","article-title":"Whose vote should count more: Optimal integration of labels from labelers of unknown expertise","volume":"22","author":"Whitehill J.","year":"2009","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_5_31_1","doi-asserted-by":"crossref","unstructured":"Yan T. V.Kumar et al. (2010).CrowdSearch: exploiting crowds for accurate real\u2010time image search on mobile phones In Proc. MobiSys'10 ACM New York NY.","DOI":"10.1145\/1814433.1814443"},{"key":"e_1_2_5_32_1","first-page":"35","volume-title":"CSDM'11","author":"Zuccon G.","year":"2011"}],"container-title":["Proceedings of the American Society for Information Science and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fmeet.14504901050","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1002%2Fmeet.14504901050","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/asistdl.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/meet.14504901050","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,20]],"date-time":"2025-10-20T11:32:54Z","timestamp":1760959974000},"score":1,"resource":{"primary":{"URL":"https:\/\/asistdl.onlinelibrary.wiley.com\/doi\/10.1002\/meet.14504901050"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,1]]},"references-count":31,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2012,1]]}},"alternative-id":["10.1002\/meet.14504901050"],"URL":"https:\/\/doi.org\/10.1002\/meet.14504901050","archive":["Portico"],"relation":{},"ISSN":["0044-7870","1550-8390"],"issn-type":[{"type":"print","value":"0044-7870"},{"type":"electronic","value":"1550-8390"}],"subject":[],"published":{"date-parts":[[2012,1]]}}}