{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,29]],"date-time":"2025-09-29T08:15:31Z","timestamp":1759133731802,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":27,"publisher":"ACM","license":[{"start":{"date-parts":[[2013,2,23]],"date-time":"2013-02-23T00:00:00Z","timestamp":1361577600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2013,2,23]]},"DOI":"10.1145\/2441776.2441848","type":"proceedings-article","created":{"date-parts":[[2013,2,22]],"date-time":"2013-02-22T19:25:33Z","timestamp":1361561133000},"page":"649-660","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":35,"title":["Quality control mechanisms for crowdsourcing"],"prefix":"10.1145","author":[{"given":"Derek L.","family":"Hansen","sequence":"first","affiliation":[{"name":"Brigham Young University, Provo, Utah, USA"}]},{"given":"Patrick J.","family":"Schone","sequence":"additional","affiliation":[{"name":"FamilySearch, Salt Lake City, Utah, USA"}]},{"given":"Douglas","family":"Corey","sequence":"additional","affiliation":[{"name":"Brigham Young University, Provo, Utah, USA"}]},{"given":"Matthew","family":"Reid","sequence":"additional","affiliation":[{"name":"Brigham Young University, Provo, Utah, USA"}]},{"given":"Jake","family":"Gehring","sequence":"additional","affiliation":[{"name":"FamilySearch, Salt Lake City, Utah, USA"}]}],"member":"320","published-online":{"date-parts":[[2013,2,23]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Cognitive psychology and its implications","author":"Anderson J.","year":"2009","unstructured":"Anderson , J. Cognitive psychology and its implications , 7 th edition. Worth Publishers , 2009 . Anderson, J. Cognitive psychology and its implications, 7th edition. Worth Publishers, 2009.","edition":"7"},{"key":"e_1_3_2_1_2_1","volume-title":"Proceedings of the 2011 Workshop on Historical Document Imaging and Processing, ACM Press","author":"Barret B.","year":"2011","unstructured":"Barret , B. , Brown , M.S. , Manmatha , R. , and Gehring , J . Proceedings of the 2011 Workshop on Historical Document Imaging and Processing, ACM Press ( 2011 ). Barret, B., Brown, M.S., Manmatha, R., and Gehring, J. Proceedings of the 2011 Workshop on Historical Document Imaging and Processing, ACM Press (2011)."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1866029.1866078"},{"key":"e_1_3_2_1_4_1","unstructured":"Chrons O. and Sundell S. Digitalkoot: Making old archives accessible using crowdsourcing. Human Computation: Papers from the 2011 AAAI Workshop (2011).  Chrons O. and Sundell S. Digitalkoot: Making old archives accessible using crowdsourcing. Human Computation: Papers from the 2011 AAAI Workshop (2011)."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1054972.1054975"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"crossref","DOI":"10.1002\/0471728438","volume-title":"Mixed Models: Theory and Applications","author":"Demidenko E.","year":"2004","unstructured":"Demidenko , E. Mixed Models: Theory and Applications . Wiley-Interscience , 2004 . Demidenko, E. Mixed Models: Theory and Applications. Wiley-Interscience, 2004."},{"key":"e_1_3_2_1_7_1","volume-title":"Proc. of the 5th Linguistic Annotation Workshop (LAW V '11)","author":"Dligach D.","year":"2011","unstructured":"Dligach , D. , and Palmer , M . 2011. Reducing the need for double annotation . In Proc. of the 5th Linguistic Annotation Workshop (LAW V '11) , ( 2011 ), 65--73. Dligach, D., and Palmer, M. 2011. Reducing the need for double annotation. In Proc. of the 5th Linguistic Annotation Workshop (LAW V '11), (2011), 65--73."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1924421.1924442"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"crossref","unstructured":"Duguid P. 2006. Limits of self-organization: Peer production and \"laws of quality\". First Monday 11 (10).  Duguid P. 2006. Limits of self-organization: Peer production and \"laws of quality\". First Monday 11 (10).","DOI":"10.5210\/fm.v11i10.1405"},{"key":"e_1_3_2_1_10_1","first-page":"10","article-title":"Towards an integrated crowdsourcing definition","volume":"20","author":"Estell\u00e9s-Arolas E.","year":"2012","unstructured":"Estell\u00e9s-Arolas , E. , and Gonz\u00e1lez-Ladr\u00f3n -de-Guevara, F . Towards an integrated crowdsourcing definition . J. of Information Science , 20 , 10 ( 2012 ), 1--14. Estell\u00e9s-Arolas, E., and Gonz\u00e1lez-Ladr\u00f3n-de-Guevara, F. Towards an integrated crowdsourcing definition. J. of Information Science, 20, 10 (2012), 1--14.","journal-title":"J. of Information Science"},{"key":"e_1_3_2_1_11_1","volume-title":"Proc. SocialCom '11","author":"Hansen D.","year":"2011","unstructured":"1 Hansen , D. , Jacobs , D. , Lewis , D. , Biswas , A. , Preece , J. , Rotman , D. , and Stevens , E . Odd Leaf Out: Improv-ing visual recognition with games . In Proc. SocialCom '11 , IEEE ( 2011 ), 87--94. 1Hansen, D., Jacobs, D., Lewis, D., Biswas, A., Preece, J., Rotman, D., and Stevens, E. Odd Leaf Out: Improv-ing visual recognition with games. In Proc. SocialCom '11, IEEE (2011), 87--94."},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/HICSS.2009.137"},{"key":"e_1_3_2_1_13_1","unstructured":"Howe J. Crowdsourcing: Why the Power of the Crowd is Driving the Future of Business http:\/\/www.crowdsourcing.com\/   Howe J. Crowdsourcing: Why the Power of the Crowd is Driving the Future of Business http:\/\/www.crowdsourcing.com\/"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.5555\/1564131.1564137"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1460563.1460572"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/8472.001.0001","volume-title":"Building Successful Online Communities: Evidence-Based Social Design","author":"Kraut R.E.","year":"2012","unstructured":"Kraut , R.E. and Resnick , P . Building Successful Online Communities: Evidence-Based Social Design . The MIT Press , 2012 . Kraut, R.E. and Resnick, P. Building Successful Online Communities: Evidence-Based Social Design. The MIT Press, 2012."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1083-6101.2005.tb00273.x"},{"key":"e_1_3_2_1_18_1","volume-title":"Workshop on Crowdsourcing and Human Computation, Services, Studies and Platforms, ACM CHI","author":"Little G.","year":"2011","unstructured":"Little , G. and Sun , Y . Human OCR: insights from a complex human computation process . Workshop on Crowdsourcing and Human Computation, Services, Studies and Platforms, ACM CHI , ( 2011 ). Little, G. and Sun, Y. Human OCR: insights from a complex human computation process. Workshop on Crowdsourcing and Human Computation, Services, Studies and Platforms, ACM CHI, (2011)."},{"key":"e_1_3_2_1_19_1","volume-title":"International Workshop on the Web and Database","author":"McCann R.","year":"2003","unstructured":"McCann , R. , Doan , A , Varadajan , V. , and Kramnik , A . Building Data Integration Systems via Mass Collaboration . In International Workshop on the Web and Database , ( 2003 ). McCann, R., Doan, A, Varadajan, V., and Kramnik, A. Building Data Integration Systems via Mass Collaboration. In International Workshop on the Web and Database, (2003)."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1743384.1743478"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979148"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.v59:6"},{"key":"e_1_3_2_1_23_1","unstructured":"Sun Y. Roy S. and Little G. D. Beyond independent agreement: a tournament selection approach for quality assurance of human computation tasks. Papers from the 2011 AAAI Workshop on Human Computation (2011).  Sun Y. Roy S. and Little G. D. Beyond independent agreement: a tournament selection approach for quality assurance of human computation tasks. Papers from the 2011 AAAI Workshop on Human Computation (2011)."},{"key":"e_1_3_2_1_24_1","volume-title":"Proc. ICDAR '07","author":"Toselli A.H.","year":"2007","unstructured":"Toselli , A.H. , Romero , V. , and Vidal , E . Rodriguez, L. Computer Assisted Transcription of Handwritten Text Images . Proc. ICDAR '07 . IEEE ( 2007 ), 944--948. Toselli, A.H., Romero, V., and Vidal, E. Rodriguez, L. Computer Assisted Transcription of Handwritten Text Images. Proc. ICDAR '07. IEEE (2007), 944--948."},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2006.196"},{"key":"e_1_3_2_1_26_1","volume-title":"reCAPTCHA: Human-based character recognition via web security measures. Science","author":"von Ahn L.","year":"2008","unstructured":"von Ahn , L. , Maurer , B. , McMillen , C. , Abraham , D. , and Blum , M . reCAPTCHA: Human-based character recognition via web security measures. Science ( 2008 ),1465--1468. von Ahn, L., Maurer, B., McMillen, C., Abraham, D., and Blum, M. reCAPTCHA: Human-based character recognition via web security measures. Science (2008),1465--1468."},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/1386790.1386837"}],"event":{"name":"CSCW '13: Computer Supported Cooperative Work","sponsor":["SIGCHI ACM Special Interest Group on Computer-Human Interaction"],"location":"San Antonio Texas USA","acronym":"CSCW '13"},"container-title":["Proceedings of the 2013 conference on Computer supported cooperative work"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2441776.2441848","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2441776.2441848","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T08:19:14Z","timestamp":1750234754000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2441776.2441848"}},"subtitle":["peer review, arbitration, & expertise at familysearch indexing"],"short-title":[],"issued":{"date-parts":[[2013,2,23]]},"references-count":27,"alternative-id":["10.1145\/2441776.2441848","10.1145\/2441776"],"URL":"https:\/\/doi.org\/10.1145\/2441776.2441848","relation":{},"subject":[],"published":{"date-parts":[[2013,2,23]]},"assertion":[{"value":"2013-02-23","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}