{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:34:16Z","timestamp":1750221256277,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":41,"publisher":"ACM","license":[{"start":{"date-parts":[[2018,6,26]],"date-time":"2018-06-26T00:00:00Z","timestamp":1529971200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1651487"],"award-info":[{"award-number":["1651487"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2018,6,26]]},"DOI":"10.1145\/3231644.3231666","type":"proceedings-article","created":{"date-parts":[[2018,6,19]],"date-time":"2018-06-19T12:36:46Z","timestamp":1529411806000},"page":"1-10","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["The potential for scientific outreach and learning in mechanical turk experiments"],"prefix":"10.1145","author":[{"given":"Eunice","family":"Jun","sequence":"first","affiliation":[{"name":"University of Washington"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Morelle","family":"Arian","sequence":"additional","affiliation":[{"name":"University of Washington"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Katharina","family":"Reinecke","sequence":"additional","affiliation":[{"name":"University of Washington"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2018,6,26]]},"reference":[{"key":"e_1_3_2_1_1_1","unstructured":"American Psychological Association. 2017. Ethical Principles of Psychologists and Code of Conduct. (2017).  American Psychological Association. 2017. Ethical Principles of Psychologists and Code of Conduct. (2017)."},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1979742.1979606"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1093\/pan\/mpr057"},{"key":"e_1_3_2_1_4_1","volume-title":"Taxonomy of Educational Objectives: The Classification of Educational Goals. Handbook I: Cognitive Domain","author":"Bloom Benjamin S","year":"1972","unstructured":"Benjamin S Bloom , Max D Engelhart , Edward J Furst , Walker H Hill , David R Krathwohl , and others. 1956. Taxonomy of Educational Objectives: The Classification of Educational Goals. Handbook I: Cognitive Domain . New York : David McKay Company . Inc.(7th Edition 1972 ) (1956). Benjamin S Bloom, Max D Engelhart, Edward J Furst, Walker H Hill, David R Krathwohl, and others. 1956. Taxonomy of Educational Objectives: The Classification of Educational Goals. Handbook I: Cognitive Domain. New York: David McKay Company. Inc.(7th Edition 1972) (1956)."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1083-6101.2007.00343.x"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858198"},{"key":"e_1_3_2_1_7_1","first-page":"1","article-title":"brms: An R package for Bayesian multilevel models using Stan","volume":"80","author":"Buerkner Paul-Christian","year":"2016","unstructured":"Paul-Christian Buerkner and others. 2016 . brms: An R package for Bayesian multilevel models using Stan . Journal of Statistical Software 80 , 1 (2016), 1 -- 28 . Paul-Christian Buerkner and others. 2016. brms: An R package for Bayesian multilevel models using Stan. Journal of Statistical Software 80, 1 (2016), 1--28.","journal-title":"Journal of Statistical Software"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1177\/1745691610393980"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025874"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0057410"},{"key":"e_1_3_2_1_11_1","volume-title":"Manual for kit of factor-referenced cognitive tests","author":"Ekstrom Ruth B","year":"1976","unstructured":"Ruth B Ekstrom , John W French , Harry H Harman , and Diran Dermen . 1976. Manual for kit of factor-referenced cognitive tests . Princeton, NJ : Educational testing service ( 1976 ). Ruth B Ekstrom, John W French, Harry H Harman, and Diran Dermen. 1976. Manual for kit of factor-referenced cognitive tests. Princeton, NJ: Educational testing service (1976)."},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2815585.2815739"},{"key":"e_1_3_2_1_13_1","volume-title":"Is the Web as good as the lab? Comparable performance from Web and lab in cognitive\/perceptual experiments. Psychonomic bulletin & review 19, 5","author":"Germine Laura","year":"2012","unstructured":"Laura Germine , Ken Nakayama , Bradley C Duchaine , Christopher F Chabris , Garga Chatterjee , and Jeremy B Wilmer . 2012. Is the Web as good as the lab? Comparable performance from Web and lab in cognitive\/perceptual experiments. Psychonomic bulletin & review 19, 5 ( 2012 ), 847--857. Laura Germine, Ken Nakayama, Bradley C Duchaine, Christopher F Chabris, Garga Chatterjee, and Jeremy B Wilmer. 2012. Is the Web as good as the lab? Comparable performance from Web and lab in cognitive\/perceptual experiments. Psychonomic bulletin & review 19, 5 (2012), 847--857."},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173574.3174023"},{"key":"e_1_3_2_1_15_1","volume-title":"The online laboratory: Conducting experiments in a real labor market. Experimental Economics","author":"Horton J J","year":"2011","unstructured":"J J Horton , D G Rand , and R J Zeckhauser . 2011. The online laboratory: Conducting experiments in a real labor market. Experimental Economics ( 2011 ). J J Horton, D G Rand, and R J Zeckhauser. 2011. The online laboratory: Conducting experiments in a real labor market. Experimental Economics (2011)."},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818048.2820024"},{"key":"e_1_3_2_1_17_1","unstructured":"Panagiotis G Ipeirotis. 2010. Demographics of mechanical turk. (2010).  Panagiotis G Ipeirotis. 2010. Demographics of mechanical turk. (2010)."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2470742"},{"key":"e_1_3_2_1_19_1","unstructured":"Eunice Jun Gary Hsieh and Katharina Reinecke. 2017. Types of motivation affect study selection attention and dropouts in online experiments. In CSCW.  Eunice Jun Gary Hsieh and Katharina Reinecke. 2017. Types of motivation affect study selection attention and dropouts in online experiments. In CSCW."},{"key":"e_1_3_2_1_20_1","first-page":"1","article-title":"More than fun and money. Worker Motivation in Crowdsourcing-A Study on Mechanical Turk","volume":"11","author":"Kaufmann Nicolas","year":"2011","unstructured":"Nicolas Kaufmann , Thimo Schulze , and Daniel Veit . 2011 . More than fun and money. Worker Motivation in Crowdsourcing-A Study on Mechanical Turk .. In AMCIS , Vol. 11. 1 -- 11 . Nicolas Kaufmann, Thimo Schulze, and Daniel Veit. 2011. More than fun and money. Worker Motivation in Crowdsourcing-A Study on Mechanical Turk.. In AMCIS, Vol. 11. 1--11.","journal-title":"AMCIS"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025592"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2441776.2441923"},{"key":"e_1_3_2_1_23_1","volume-title":"Psychological research online: Report of Board of Scientific Affairs' Advisory Group on the conduct of research on the internet. American psychologist 59, 2","author":"Kraut Robert","year":"2004","unstructured":"Robert Kraut , Judith Olson , Mahzarin Banaji , Amy Bruckman , Jeffrey Cohen , and Mick Couper . 2004. Psychological research online: Report of Board of Scientific Affairs' Advisory Group on the conduct of research on the internet. American psychologist 59, 2 ( 2004 ), 105. Robert Kraut, Judith Olson, Mahzarin Banaji, Amy Bruckman, Jeffrey Cohen, and Mick Couper. 2004. Psychological research online: Report of Board of Scientific Affairs' Advisory Group on the conduct of research on the internet. American psychologist 59, 2 (2004), 105."},{"key":"e_1_3_2_1_24_1","unstructured":"Siou Chew Kuek Cecilia Paradi-Guilford Toks Fayomi Saori Imaizumi Panos Ipeirotis Patricia Pina and Manpreet Singh. 2015. The global opportunity in online outsourcing. (2015).  Siou Chew Kuek Cecilia Paradi-Guilford Toks Fayomi Saori Imaizumi Panos Ipeirotis Patricia Pina and Manpreet Singh. 2015. The global opportunity in online outsourcing. (2015)."},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2966.2008.13490.x"},{"key":"e_1_3_2_1_26_1","volume-title":"June {online}.","author":"Lane N","year":"1997","unstructured":"N Lane . 1997. An open letter to scientists and engineers: \"Let's get the word out together about why science matters.\" National Science Foundation , June {online}. ( 1997 ). N Lane. 1997. An open letter to scientists and engineers: \"Let's get the word out together about why science matters.\" National Science Foundation, June {online}. (1997)."},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3026038"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2531602.2531663"},{"key":"e_1_3_2_1_29_1","volume-title":"Conducting behavioral research on Amazon's Mechanical Turk. Behavior research methods 44, 1","author":"Mason Winter","year":"2012","unstructured":"Winter Mason and Siddharth Suri . 2012. Conducting behavioral research on Amazon's Mechanical Turk. Behavior research methods 44, 1 ( 2012 ), 1--23. Winter Mason and Siddharth Suri. 2012. Conducting behavioral research on Amazon's Mechanical Turk. Behavior research methods 44, 1 (2012), 1--23."},{"key":"e_1_3_2_1_30_1","volume-title":"https:\/\/docs.google.com\/spreadsheets\/d\/1T3yP_Jo4qELrwsE2NAPNs67L1AWmpAEr9vnhreGJ-K0\/edit#gid=1993674859, last accessed","author":"Milland Kristy","year":"2015","unstructured":"Kristy Milland . 2015. ( 2015 ). https:\/\/docs.google.com\/spreadsheets\/d\/1T3yP_Jo4qELrwsE2NAPNs67L1AWmpAEr9vnhreGJ-K0\/edit#gid=1993674859, last accessed January 18, 2018. Kristy Milland. 2015. (2015). https:\/\/docs.google.com\/spreadsheets\/d\/1T3yP_Jo4qELrwsE2NAPNs67L1AWmpAEr9vnhreGJ-K0\/edit#gid=1993674859, last accessed January 18, 2018."},{"key":"e_1_3_2_1_31_1","unstructured":"Social Psychology Network. 2017. (2017). http:\/\/www.socialpsychology.org last accessed August 20 2017.  Social Psychology Network. 2017. (2017). http:\/\/www.socialpsychology.org last accessed August 20 2017."},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025473"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025769"},{"key":"e_1_3_2_1_34_1","volume-title":"Running experiments on Amazon Mechanical Turk. Judgment and Decision Making 5, 5","author":"Paolacci Gabriele","year":"2010","unstructured":"Gabriele Paolacci , Jesse Chandler , and Panagiotis G Ipeirotis . 2010. Running experiments on Amazon Mechanical Turk. Judgment and Decision Making 5, 5 ( 2010 ). Gabriele Paolacci, Jesse Chandler, and Panagiotis G Ipeirotis. 2010. Running experiments on Amazon Mechanical Turk. Judgment and Decision Making 5, 5 (2010)."},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2675133.2675246"},{"key":"e_1_3_2_1_36_1","unstructured":"SciStarter. 2017. (2017). http:\/\/www.scistarter.com last accessed August 20 2017.  SciStarter. 2017. (2017). http:\/\/www.scistarter.com last accessed August 20 2017."},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/1837885.1837891"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.biocon.2009.05.006"},{"key":"e_1_3_2_1_39_1","volume-title":"Geza Kovacs, Andreas Veit, Ranjay Krishna, Imanol Arrieta Ibarra, Camelia Simoiu, Michael Wilber, Serge Belongie, Sharad Goel, and others.","author":"Vaish Rajan","year":"2017","unstructured":"Rajan Vaish , Snehalkumar Neil S Gaikwad , Geza Kovacs, Andreas Veit, Ranjay Krishna, Imanol Arrieta Ibarra, Camelia Simoiu, Michael Wilber, Serge Belongie, Sharad Goel, and others. 2017 . Crowd Research : Open and scalable university laboratories. (2017). Rajan Vaish, Snehalkumar Neil S Gaikwad, Geza Kovacs, Andreas Veit, Ranjay Krishna, Imanol Arrieta Ibarra, Camelia Simoiu, Michael Wilber, Serge Belongie, Sharad Goel, and others. 2017. Crowd Research: Open and scalable university laboratories. (2017)."},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/HICSS.2011.207"},{"key":"e_1_3_2_1_41_1","volume-title":"The pitfall of experimenting on the web: How unattended selective attrition leads to surprising (yet false) research conclusions","author":"Zhou Haotian","year":"2016","unstructured":"Haotian Zhou and Ayelet Fishbach . 2016. The pitfall of experimenting on the web: How unattended selective attrition leads to surprising (yet false) research conclusions . American Psychological Association ( 2016 ). Haotian Zhou and Ayelet Fishbach. 2016. The pitfall of experimenting on the web: How unattended selective attrition leads to surprising (yet false) research conclusions. American Psychological Association (2016)."}],"event":{"name":"L@S '18: Fifth (2018) ACM Conference on Learning @ Scale","acronym":"L@S '18","location":"London United Kingdom"},"container-title":["Proceedings of the Fifth Annual ACM Conference on Learning at Scale"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3231644.3231666","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3231644.3231666","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3231644.3231666","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T02:08:08Z","timestamp":1750212488000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3231644.3231666"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,6,26]]},"references-count":41,"alternative-id":["10.1145\/3231644.3231666","10.1145\/3231644"],"URL":"https:\/\/doi.org\/10.1145\/3231644.3231666","relation":{},"subject":[],"published":{"date-parts":[[2018,6,26]]},"assertion":[{"value":"2018-06-26","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}