{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T21:10:23Z","timestamp":1778620223188,"version":"3.51.4"},"publisher-location":"New York, NY, USA","reference-count":53,"publisher":"ACM","license":[{"start":{"date-parts":[[2019,5,13]],"date-time":"2019-05-13T00:00:00Z","timestamp":1557705600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2019,5,13]]},"DOI":"10.1145\/3308558.3313554","type":"proceedings-article","created":{"date-parts":[[2019,5,13]],"date-time":"2019-05-13T12:17:59Z","timestamp":1557749879000},"page":"1794-1805","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Leveraging Peer Communication to Enhance Crowdsourcing"],"prefix":"10.1145","author":[{"given":"Wei","family":"Tang","sequence":"first","affiliation":[{"name":"Washington University in St. Louis, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ming","family":"Yin","sequence":"additional","affiliation":[{"name":"Purdue University, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chien-Ju","family":"Ho","sequence":"additional","affiliation":[{"name":"Washington University in St. Louis, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2019,5,13]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3209542.3209558"},{"key":"e_1_3_2_1_2_1","volume-title":"Optimally interacting minds. Science329, 5995","author":"Bahrami Bahador","year":"2010","unstructured":"Bahador Bahrami , Karsten Olsen , Peter E Latham , Andreas Roepstorff , Geraint Rees , and Chris D Frith . 2010. Optimally interacting minds. Science329, 5995 ( 2010 ), 1081-1085. Bahador Bahrami, Karsten Olsen, Peter E Latham, Andreas Roepstorff, Geraint Rees, and Chris D Frith. 2010. Optimally interacting minds. Science329, 5995 (2010), 1081-1085."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2047196.2047201"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/1866029.1866078"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1866029.1866080"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858237"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3026044"},{"key":"e_1_3_2_1_8_1","volume-title":"International Conference on Machine Learning (ICML).","author":"Chen Xi","year":"2013","unstructured":"Xi Chen , Qihang Lin , and Dengyong Zhou . 2013 . Optimistic knowledge gradient policy for optimal budget allocation in crowdsourcing . In International Conference on Machine Learning (ICML). Xi Chen, Qihang Lin, and Dengyong Zhou. 2013. Optimistic knowledge gradient policy for optimal budget allocation in crowdsourcing. In International Conference on Machine Learning (ICML)."},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2466265"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICTAI.2008.141"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2675133.2675251"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1119\/1.1374249"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2013.06.002"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"crossref","unstructured":"A. P. Dawid and A. M. Skene. 1979. Maximum likeihood estimation of observer error-rates using the EM algorithm. Applied Statistics28(1979) 20-28. A. P. Dawid and A. M. Skene. 1979. Maximum likeihood estimation of observer error-rates using the EM algorithm. Applied Statistics28(1979) 20-28.","DOI":"10.2307\/2346806"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1111\/j.2517-6161.1977.tb01600.x","article-title":"Maximum likelihood from incomplete data via the EM algorithm","volume":"39","author":"Dempster Arthur P.","year":"1977","unstructured":"Arthur P. Dempster , Nan M. Laird , and Donald B. Rubin . 1977 . Maximum likelihood from incomplete data via the EM algorithm . Journal of the Royal Statistical Society: Series B39 (1977), 1 - 38 . Arthur P. Dempster, Nan M. Laird, and Donald B. Rubin. 1977. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society: Series B39 (1977), 1-38.","journal-title":"Journal of the Royal Statistical Society: Series"},{"key":"e_1_3_2_1_16_1","volume-title":"Improved learning in a large-enrollment physics class. science332, 6031","author":"Deslauriers Louis","year":"2011","unstructured":"Louis Deslauriers , Ellen Schelew , and Carl Wieman . 2011. Improved learning in a large-enrollment physics class. science332, 6031 ( 2011 ), 862-864. Louis Deslauriers, Ellen Schelew, and Carl Wieman. 2011. Improved learning in a large-enrollment physics class. science332, 6031 (2011), 862-864."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2736277.2741685"},{"key":"e_1_3_2_1_18_1","volume-title":"Fourth AAAI Conference on Human Computation and Crowdsourcing (HCOMP).","author":"Drapeau Ryan","unstructured":"Ryan Drapeau , Lydia B. Chilton , Jonathan Bragg , and Daniel S. Weld . 2016. MicroTalk: Using Argumentation to Improve Crowdsourcing Accuracy . In Fourth AAAI Conference on Human Computation and Crowdsourcing (HCOMP). Ryan Drapeau, Lydia B. Chilton, Jonathan Bragg, and Daniel S. Weld. 2016. MicroTalk: Using Argumentation to Improve Crowdsourcing Accuracy. In Fourth AAAI Conference on Human Computation and Crowdsourcing (HCOMP)."},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818048.2819942"},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2660398.2660403"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025781"},{"key":"e_1_3_2_1_22_1","volume-title":"Adaptive Task Assignment for Crowdsourced Classification. In The 30th International Conference on Machine Learning (ICML).","author":"Ho Chien-Ju","year":"2013","unstructured":"Chien-Ju Ho , Shahin Jabbari , and Jennifer Wortman Vaughan . 2013 . Adaptive Task Assignment for Crowdsourced Classification. In The 30th International Conference on Machine Learning (ICML). Chien-Ju Ho, Shahin Jabbari, and Jennifer Wortman Vaughan. 2013. Adaptive Task Assignment for Crowdsourced Classification. In The 30th International Conference on Machine Learning (ICML)."},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2736277.2741102"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1807342.1807376"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2470654.2470742"},{"key":"e_1_3_2_1_26_1","unstructured":"Rong Jin and Zoubin Ghahramani. 2003. Learning with multiple labels. In Advances in Neural Information Processing Systems (NIPS). Rong Jin and Zoubin Ghahramani. 2003. Learning with multiple labels. In Advances in Neural Information Processing Systems (NIPS)."},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/Allerton.2011.6120180"},{"key":"e_1_3_2_1_28_1","volume-title":"The 25th Annual Conference on Neural Information Processing Systems (NIPS).","author":"Karger R.","year":"2011","unstructured":"David. R. Karger , Sewoong Oh , and Devavrat Shah . 2011 . Iterative learning for reliable crowdsourcing systems . In The 25th Annual Conference on Neural Information Processing Systems (NIPS). David. R. Karger, Sewoong Oh, and Devavrat Shah. 2011. Iterative learning for reliable crowdsourcing systems. In The 25th Annual Conference on Neural Information Processing Systems (NIPS)."},{"key":"e_1_3_2_1_29_1","volume-title":"Novel Dataset for Fine-Grained Image Categorization. In First Workshop on Fine-Grained Visual Categorization, IEEE Conference on Computer Vision and Pattern Recognition","author":"Khosla Aditya","year":"2011","unstructured":"Aditya Khosla , Nityananda Jayadevaprakash , Bangpeng Yao , and Li Fei-Fei . 2011 . Novel Dataset for Fine-Grained Image Categorization. In First Workshop on Fine-Grained Visual Categorization, IEEE Conference on Computer Vision and Pattern Recognition . Colorado Springs, CO. Aditya Khosla, Nityananda Jayadevaprakash, Bangpeng Yao, and Li Fei-Fei. 2011. Novel Dataset for Fine-Grained Image Categorization. In First Workshop on Fine-Grained Visual Categorization, IEEE Conference on Computer Vision and Pattern Recognition. Colorado Springs, CO."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2556288.2556986"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2047196.2047202"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2145204.2145354"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858144"},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2835776.2835797"},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/1866029.1866040"},{"key":"e_1_3_2_1_36_1","unstructured":"Frank Lyman. 1987. Think-Pair-Share: An Expanding Teaching Technique. MAA-CIE Cooperative News(1987). Frank Lyman. 1987. Think-Pair-Share: An Expanding Teaching Technique. MAA-CIE Cooperative News(1987)."},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2531602.2531663"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/1600150.1600175"},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858490"},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2047196.2047198"},{"key":"e_1_3_2_1_41_1","volume-title":"Learning from crowds. Journal of Machine Learning Research11","author":"Raykar Vikas","year":"2010","unstructured":"Vikas Raykar , Shipeng Yu , Linda Zhao , Gerardo Valadez , Charles Florin , Luca Bogoni , and Linda Moy . 2010. Learning from crowds. Journal of Machine Learning Research11 ( 2010 ), 1297-1322. Vikas Raykar, Shipeng Yu, Linda Zhao, Gerardo Valadez, Charles Florin, Luca Bogoni, and Linda Moy. 2010. Learning from crowds. Journal of Machine Learning Research11 (2010), 1297-1322."},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2642918.2647409"},{"key":"e_1_3_2_1_43_1","unstructured":"Jakob Rogstadius Vassilis Kostakos Aniket Kittur Boris Smus Jim Laredo and Maja Vukovic. 2011. An assessment of intrinsic and extrinsic motivation on task performance in crowdsourcing markets.ICWSM11(2011) 17-21. Jakob Rogstadius Vassilis Kostakos Aniket Kittur Boris Smus Jim Laredo and Maja Vukovic. 2011. An assessment of intrinsic and extrinsic motivation on task performance in crowdsourcing markets.ICWSM11(2011) 17-21."},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702508"},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3274423"},{"key":"e_1_3_2_1_46_1","volume-title":"The 29th Annual Conference on Neural Information Processing Systems (NIPS).","author":"Shah Nihar Bhadresh","year":"2015","unstructured":"Nihar Bhadresh Shah and Denny Zhou . 2015 . Double or Nothing: Multiplicative Incentive Mechanisms for Crowdsourcing . In The 29th Annual Conference on Neural Information Processing Systems (NIPS). Nihar Bhadresh Shah and Denny Zhou. 2015. Double or Nothing: Multiplicative Incentive Mechanisms for Crowdsourcing. In The 29th Annual Conference on Neural Information Processing Systems (NIPS)."},{"key":"e_1_3_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/1958824.1958865"},{"key":"e_1_3_2_1_48_1","volume-title":"Why peer discussion improves student performance on in-class concept questions. Science323, 5910","author":"Smith Michelle K","year":"2009","unstructured":"Michelle K Smith , William B Wood , Wendy K Adams , Carl Wieman , Jennifer K Knight , Nancy Guild , and Tin Tin Su. 2009. Why peer discussion improves student performance on in-class concept questions. Science323, 5910 ( 2009 ), 122-124. Michelle K Smith, William B Wood, Wendy K Adams, Carl Wieman, Jennifer K Knight, Nancy Guild, and Tin Tin Su. 2009. Why peer discussion improves student performance on in-class concept questions. Science323, 5910 (2009), 122-124."},{"key":"e_1_3_2_1_49_1","volume-title":"Jacob Bergsma, Javier R. Movellan, and Paul L. Ruvolo.","author":"Whitehill Jacob","year":"2009","unstructured":"Jacob Whitehill , Ting fan Wu , Jacob Bergsma, Javier R. Movellan, and Paul L. Ruvolo. 2009 . Whose vote should count more: Optimal integration of labels from labelers of unknown expertise. In Advances in Neural Information Processing Systems (NIPS) . Jacob Whitehill, Ting fan Wu, Jacob Bergsma, Javier R. Movellan, and Paul L. Ruvolo. 2009. Whose vote should count more: Optimal integration of labels from labelers of unknown expertise. In Advances in Neural Information Processing Systems (NIPS)."},{"key":"e_1_3_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.5555\/2891460.2891626"},{"key":"e_1_3_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/2872427.2883036"},{"key":"e_1_3_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2531602.2531729"},{"key":"e_1_3_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.14778\/3055540.3055547"}],"event":{"name":"WWW '19: The Web Conference","location":"San Francisco CA USA","acronym":"WWW '19","sponsor":["IW3C2 International World Wide Web Conference Committee"]},"container-title":["The World Wide Web Conference"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3308558.3313554","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3308558.3313554","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:53:20Z","timestamp":1750204400000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3308558.3313554"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,5,13]]},"references-count":53,"alternative-id":["10.1145\/3308558.3313554","10.1145\/3308558"],"URL":"https:\/\/doi.org\/10.1145\/3308558.3313554","relation":{},"subject":[],"published":{"date-parts":[[2019,5,13]]},"assertion":[{"value":"2019-05-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}