{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T06:59:29Z","timestamp":1772780369454,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":50,"publisher":"ACM","license":[{"start":{"date-parts":[[2017,5,2]],"date-time":"2017-05-02T00:00:00Z","timestamp":1493683200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2017,5,2]]},"DOI":"10.1145\/3025453.3025473","type":"proceedings-article","created":{"date-parts":[[2017,5,3]],"date-time":"2017-05-03T12:34:28Z","timestamp":1493814868000},"page":"6800-6812","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["Citizen Science Opportunities in Volunteer-Based Online Experiments"],"prefix":"10.1145","author":[{"given":"Nigini","family":"Oliveira","sequence":"first","affiliation":[{"name":"University of Washington & Universidade Federal de Campina Grande, Seattle, WA, USA"}]},{"given":"Eunice","family":"Jun","sequence":"additional","affiliation":[{"name":"University of Washington, Seattle, WA, USA"}]},{"given":"Katharina","family":"Reinecke","sequence":"additional","affiliation":[{"name":"University of Washington, Seattle, WA, USA"}]}],"member":"320","published-online":{"date-parts":[[2017,5,2]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.3758\/BRM.40.4.1111"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1111\/1540-4560.00269"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1525\/bio.2009.59.11.9"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1080\/09500690500069483"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.333.6039.173"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature09304"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1978942.1979213"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1080\/1533015X.2011.603611"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1002\/prot.21636"},{"key":"e_1_3_2_1_10_1","first-page":"6","article-title":"Why do people write for Wikipedia? Incentives to contribute to open--content publishing","volume":"5","author":"Forte Andrea","year":"2005","unstructured":"Andrea Forte and Amy Bruckman . 2005 . Why do people write for Wikipedia? Incentives to contribute to open--content publishing . Proc. of GROUP 5 (2005), 6 -- 9 . Andrea Forte and Amy Bruckman. 2005. Why do people write for Wikipedia? Incentives to contribute to open--content publishing. Proc. of GROUP 5 (2005), 6--9.","journal-title":"Proc. of GROUP"},{"key":"e_1_3_2_1_11_1","volume-title":"Proceedings of the 29th Annual ACM Symposium on User Interface Software & Technology (UIST '16)","author":"Gaikwad Snehalkumar","unstructured":"Snehalkumar (Neil) S. Gaikwad , Durim Morina , Adam Ginzberg , Catherine Mullings , Shirish Goyal , Dilrukshi Gamage , Christopher Diemert , Mathias Burton , Sharon Zhou , Mark Whiting , Karolina Ziulkoski , Alipta Ballav , Aaron Gilbee , Senadhipathige S. Niranga , Vibhor Sehgal , Jasmine Lin , Leonardy Kristianto , Angela Richmond-Fuller , Jeff Regino , Nalin Chhibber , Dinesh Majeti , Sachin Sharma , Kamila Mananova , Dinesh Dhakal , William Dai , Victoria Purynova , Samarth Sandeep , Varshine Chandrakanthan , Tejas Sarma , Sekandar Matin , Ahmed Nasser , Rohit Nistala , Alexander Stolzoff , Kristy Milland , Vinayak Mathur , Rajan Vaish , and Michael S. Bernstein . 2016. Boomerang: Aligning Worker and Requester Incentives on Crowdsourcing Platforms . In Proceedings of the 29th Annual ACM Symposium on User Interface Software & Technology (UIST '16) . 101--102. Snehalkumar (Neil) S. Gaikwad, Durim Morina, Adam Ginzberg, Catherine Mullings, Shirish Goyal, Dilrukshi Gamage, Christopher Diemert, Mathias Burton, Sharon Zhou, Mark Whiting, Karolina Ziulkoski, Alipta Ballav, Aaron Gilbee, Senadhipathige S. Niranga, Vibhor Sehgal, Jasmine Lin, Leonardy Kristianto, Angela Richmond-Fuller, Jeff Regino, Nalin Chhibber, Dinesh Majeti, Sachin Sharma, Kamila Mananova, Dinesh Dhakal, William Dai, Victoria Purynova, Samarth Sandeep, Varshine Chandrakanthan, Tejas Sarma, Sekandar Matin, Ahmed Nasser, Rohit Nistala, Alexander Stolzoff, Kristy Milland, Vinayak Mathur, Rajan Vaish, and Michael S. Bernstein. 2016. Boomerang: Aligning Worker and Requester Incentives on Crowdsourcing Platforms. In Proceedings of the 29th Annual ACM Symposium on User Interface Software & Technology (UIST '16). 101--102."},{"key":"e_1_3_2_1_12_1","unstructured":"GamesWithWords. 2016. (2016). http:\/\/www.gameswithwords.org last accessed September 2 2016.  GamesWithWords. 2016. (2016). http:\/\/www.gameswithwords.org last accessed September 2 2016."},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.3758\/s13423-012-0296-9"},{"key":"e_1_3_2_1_14_1","volume-title":"Massively collaborative mathematics. Nature 461, 7266","author":"Gowers Timothy","year":"2009","unstructured":"Timothy Gowers and Michael Nielsen . 2009. Massively collaborative mathematics. Nature 461, 7266 ( 2009 ), 879--881. Timothy Gowers and Michael Nielsen. 2009. Massively collaborative mathematics. Nature 461, 7266 (2009), 879--881."},{"key":"e_1_3_2_1_15_1","volume-title":"Citizen Science and Volunteered Geographic Information: Overview and Typology of Participation","author":"Haklay Muki","unstructured":"Muki Haklay . 2013. Citizen Science and Volunteered Geographic Information: Overview and Typology of Participation . Springer Netherlands , Dordrecht , 105--122. Muki Haklay. 2013. Citizen Science and Volunteered Geographic Information: Overview and Typology of Participation. Springer Netherlands, Dordrecht, 105--122."},{"key":"e_1_3_2_1_16_1","volume-title":"The weirdest people in the world? Behavioral and brain sciences 33, 2--3","author":"Henrich Joseph","year":"2010","unstructured":"Joseph Henrich , Steven J Heine , and Ara Norenzayan . 2010. The weirdest people in the world? Behavioral and brain sciences 33, 2--3 ( 2010 ), 61--83. Joseph Henrich, Steven J Heine, and Ara Norenzayan. 2010. The weirdest people in the world? Behavioral and brain sciences 33, 2--3 (2010), 61--83."},{"key":"e_1_3_2_1_17_1","volume-title":"https:\/\/implicit.harvard.edu\/implicit\/, last accessed","author":"Implicit Project","year":"2016","unstructured":"Project Implicit . 2016. ( 2016 ). https:\/\/implicit.harvard.edu\/implicit\/, last accessed September 2, 2016. Project Implicit. 2016. (2016). https:\/\/implicit.harvard.edu\/implicit\/, last accessed September 2, 2016."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.22323\/2.15030205"},{"key":"e_1_3_2_1_19_1","volume-title":"The original KJ method","author":"Kawakita Jiro","year":"1991","unstructured":"Jiro Kawakita . 1991. The original KJ method . Tokyo : Kawakita Research Institute ( 1991 ). Jiro Kawakita. 1991. The original KJ method. Tokyo: Kawakita Research Institute (1991)."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1037\/0003-066X.59.2.105"},{"key":"e_1_3_2_1_21_1","volume-title":"Building successful online communities: Evidence-based social design","author":"Kraut Robert E","unstructured":"Robert E Kraut , Paul Resnick , Sara Kiesler , Moira Burke , Yan Chen , Niki Kittur , Joseph Konstan , Yuqing Ren , and John Riedl . 2012. Building successful online communities: Evidence-based social design . Mit Press . Robert E Kraut, Paul Resnick, Sara Kiesler, Moira Burke, Yan Chen, Niki Kittur, Joseph Konstan, Yuqing Ren, and John Riedl. 2012. Building successful online communities: Evidence-based social design. Mit Press."},{"key":"e_1_3_2_1_22_1","unstructured":"LabintheWild. 2016. (2016). http:\/\/www.labinthewild.org last accessed September 18 2016.  LabintheWild. 2016. (2016). http:\/\/www.labinthewild.org last accessed September 18 2016."},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.22323\/2.15030207"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1772690.1772766"},{"key":"e_1_3_2_1_25_1","volume-title":"Retrieved","author":"Ornithology Cornell Lab","year":"2017","unstructured":"Cornell Lab of Ornithology . 2017 a. Biotracker NSF Project. Web site. (2017) . Retrieved January 05, 2017 from http:\/\/www.birds.cornell.edu\/citscitoolkit\/ projects\/biotracker-nsf-project\/. Cornell Lab of Ornithology. 2017a. Biotracker NSF Project. Web site. (2017). Retrieved January 05, 2017 from http:\/\/www.birds.cornell.edu\/citscitoolkit\/ projects\/biotracker-nsf-project\/."},{"key":"e_1_3_2_1_26_1","volume-title":"Retrieved","author":"Ornithology Cornell Lab","year":"2017","unstructured":"Cornell Lab of Ornithology . 2017 b. eBird Help Center. Web site. (2017) . Retrieved January 05, 2017 from http:\/\/help.ebird.org\/. Cornell Lab of Ornithology. 2017b. eBird Help Center. Web site. (2017). Retrieved January 05, 2017 from http:\/\/help.ebird.org\/."},{"key":"e_1_3_2_1_27_1","volume-title":"Retrieved","author":"Ornithology Cornell Lab","year":"2017","unstructured":"Cornell Lab of Ornithology . 2017 c. NestWatch Chapters. Web site. (2017) . Retrieved January 05, 2017 from http:\/\/nestwatch.org\/connect\/nestwatch-chapters\/. Cornell Lab of Ornithology. 2017c. NestWatch Chapters. Web site. (2017). Retrieved January 05, 2017 from http:\/\/nestwatch.org\/connect\/nestwatch-chapters\/."},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1177\/0963721414531598"},{"key":"e_1_3_2_1_29_1","volume-title":"Running experiments on amazon mechanical turk. Judgment and Decision making 5, 5","author":"Paolacci Gabriele","year":"2010","unstructured":"Gabriele Paolacci , Jesse Chandler , and Panagiotis G Ipeirotis . 2010. Running experiments on amazon mechanical turk. Judgment and Decision making 5, 5 ( 2010 ), 411--419. Gabriele Paolacci, Jesse Chandler, and Panagiotis G Ipeirotis. 2010. Running experiments on amazon mechanical turk. Judgment and Decision making 5, 5 (2010), 411--419."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2016.1194153"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.17705\/1thci.00005"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858575"},{"key":"e_1_3_2_1_33_1","volume-title":"Citizen science: status and research directions for the coming decade. AGB Stars and Related Phenomenastro 2010: The Astronomy and Astrophysics Decadal Survey 2010","author":"Raddick M Jordan","year":"2009","unstructured":"M Jordan Raddick , GA Bracey , Karen Carney , Geza Gyuk , Kirk Borne , John Wallin , Suzanne Jacoby , and Adler Planetarium . 2009. Citizen science: status and research directions for the coming decade. AGB Stars and Related Phenomenastro 2010: The Astronomy and Astrophysics Decadal Survey 2010 ( 2009 ), 46P. M Jordan Raddick, GA Bracey, Karen Carney, Geza Gyuk, Kirk Borne, John Wallin, Suzanne Jacoby, and Adler Planetarium. 2009. Citizen science: status and research directions for the coming decade. AGB Stars and Related Phenomenastro 2010: The Astronomy and Astrophysics Decadal Survey 2010 (2009), 46P."},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818052.2874332"},{"key":"e_1_3_2_1_35_1","volume-title":"Retrieved","year":"2017","unstructured":"Reddit. 2017 . Reddit threads on LabintheWild experiments. Web site. (2017) . Retrieved January 05, 2017 from https:\/\/www.reddit.com\/domain\/labinthewild.org\/. Reddit. 2017. Reddit threads on LabintheWild experiments. Web site. (2017). Retrieved January 05, 2017 from https:\/\/www.reddit.com\/domain\/labinthewild.org\/."},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2675133.2675246"},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2145204.2145238"},{"key":"e_1_3_2_1_38_1","unstructured":"Volunteer Science. 2016. (2016). http:\/\/www.volunteerscience.org last accessed September 17 2016.  Volunteer Science. 2016. (2016). http:\/\/www.volunteerscience.org last accessed September 17 2016."},{"key":"e_1_3_2_1_39_1","volume-title":"Public Participation in Scientific Research: a Framework for Deliberate Design","author":"Shirk Jennifer L.","year":"2012","unstructured":"Jennifer L. Shirk , Heidi L. Ballard , Candie C. Wilderman , Tina Phillips , Andrea Wiggins , Rebecca Jordan , Ellen McCallie , Matthew Minarchek , Bruce V. Lewenstein , Marianne E. Krasny , and Rick Bonney . 2012. Public Participation in Scientific Research: a Framework for Deliberate Design . Ecology and Society 17, 2 ( 2012 ). Jennifer L. Shirk, Heidi L. Ballard, Candie C. Wilderman, Tina Phillips, Andrea Wiggins, Rebecca Jordan, Ellen McCallie, Matthew Minarchek, Bruce V. Lewenstein, Marianne E. Krasny, and Rick Bonney. 2012. Public Participation in Scientific Research: a Framework for Deliberate Design. Ecology and Society 17, 2 (2012)."},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.biocon.2013.11.003"},{"key":"e_1_3_2_1_41_1","unstructured":"TestMyBrain. 2016. (2016). http:\/\/www.testmybrain.org last accessed September 15 2016.  TestMyBrain. 2016. (2016). http:\/\/www.testmybrain.org last accessed September 15 2016."},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702420"},{"key":"e_1_3_2_1_43_1","volume-title":"Thinking scientifically during participation in a citizen-science project. Science education 84, 2","author":"Trumbull Deborah J","year":"2000","unstructured":"Deborah J Trumbull , Rick Bonney , Derek Bascom , and Anna Cabral . 2000. Thinking scientifically during participation in a citizen-science project. Science education 84, 2 ( 2000 ), 265--275. Deborah J Trumbull, Rick Bonney, Derek Bascom, and Anna Cabral. 2000. Thinking scientifically during participation in a citizen-science project. Science education 84, 2 (2000), 265--275."},{"key":"e_1_3_2_1_44_1","volume-title":"2015 Collective Intelligence Conference.","author":"Vaish Rajan","year":"2015","unstructured":"Rajan Vaish , James Davis , and Michael Bernstein . 2015 . Crowdsourcing the research process . In 2015 Collective Intelligence Conference. Rajan Vaish, James Davis, and Michael Bernstein. 2015. Crowdsourcing the research process. In 2015 Collective Intelligence Conference."},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/HICSS.2011.207"},{"key":"e_1_3_2_1_46_1","volume-title":"Voyager: Exploratory analysis via faceted browsing of visualization recommendations","author":"Wongsuphasawat Kanit","year":"2016","unstructured":"Kanit Wongsuphasawat , Dominik Moritz , Anushka Anand , Jock Mackinlay , Bill Howe , and Jeffrey Heer . 2016 . Voyager: Exploratory analysis via faceted browsing of visualization recommendations . IEEE transactions on visualization and computer graphics 22, 1 (2016), 649--658. Kanit Wongsuphasawat, Dominik Moritz, Anushka Anand, Jock Mackinlay, Bill Howe, and Jeffrey Heer. 2016. Voyager: Exploratory analysis via faceted browsing of visualization recommendations. IEEE transactions on visualization and computer graphics 22, 1 (2016), 649--658."},{"key":"e_1_3_2_1_47_1","volume-title":"eBird: Engaging Birders in Science and Conservation. PLoS Biol 9, 12 (20","author":"Wood Chris","year":"2011","unstructured":"Chris Wood , Brian Sullivan , Marshall Iliff , Daniel Fink , and Steve Kelling . 2011. eBird: Engaging Birders in Science and Conservation. PLoS Biol 9, 12 (20 Dec. 2011 ). Chris Wood, Brian Sullivan, Marshall Iliff, Daniel Fink, and Steve Kelling. 2011. eBird: Engaging Birders in Science and Conservation. PLoS Biol 9, 12 (20 Dec. 2011)."},{"key":"e_1_3_2_1_48_1","first-page":"3","article-title":"Popular Science","volume":"116","author":"Xue Katherine","year":"2014","unstructured":"Katherine Xue . 2014 . Popular Science . Harvard Magazine 116 , 3 (Jan. 2014), 54--59. http:\/\/harvardmagazine.com\/2014\/01\/popular-science Katherine Xue. 2014. Popular Science. Harvard Magazine 116, 3 (Jan. 2014), 54--59. http:\/\/harvardmagazine.com\/2014\/01\/popular-science","journal-title":"Harvard Magazine"},{"key":"e_1_3_2_1_49_1","volume-title":"Retrieved","year":"2017","unstructured":"Zooniverse. 2017 a. Zooniverse Site. Web site. (2017) . Retrieved January 05, 2017 from https:\/\/www.zooniverse.org. Zooniverse. 2017a. Zooniverse Site. Web site. (2017). Retrieved January 05, 2017 from https:\/\/www.zooniverse.org."},{"key":"e_1_3_2_1_50_1","volume-title":"Retrieved","year":"2017","unstructured":"Zooniverse. 2017 b. Zooniverse Talk. Web site. (2017) . Retrieved January 05, 2017 from https:\/\/www.zooniverse.org\/talk. Zooniverse. 2017b. Zooniverse Talk. Web site. (2017). Retrieved January 05, 2017 from https:\/\/www.zooniverse.org\/talk."}],"event":{"name":"CHI '17: CHI Conference on Human Factors in Computing Systems","location":"Denver Colorado USA","acronym":"CHI '17","sponsor":["SIGCHI ACM Special Interest Group on Computer-Human Interaction"]},"container-title":["Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3025453.3025473","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3025453.3025473","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T03:50:21Z","timestamp":1750218621000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3025453.3025473"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,5,2]]},"references-count":50,"alternative-id":["10.1145\/3025453.3025473","10.1145\/3025453"],"URL":"https:\/\/doi.org\/10.1145\/3025453.3025473","relation":{},"subject":[],"published":{"date-parts":[[2017,5,2]]},"assertion":[{"value":"2017-05-02","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}