{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:16:15Z","timestamp":1750220175191,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":23,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,10,16]],"date-time":"2021-10-16T00:00:00Z","timestamp":1634342400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2019YFB1404904"],"award-info":[{"award-number":["2019YFB1404904"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,10,16]]},"DOI":"10.1145\/3503181.3503184","type":"proceedings-article","created":{"date-parts":[[2022,3,1]],"date-time":"2022-03-01T23:13:40Z","timestamp":1646176420000},"page":"12-17","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Classification Model based on Intelligence for Crowdsourced Data Labeling: Intelligence Elicitation by Information Interaction Mechanism"],"prefix":"10.1145","author":[{"given":"Joonhyup","family":"Lee","sequence":"first","affiliation":[{"name":"Department of Automation, Tsinghua University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ziyang","family":"Wang","sequence":"additional","affiliation":[{"name":"Department of Automation, Tsinghua University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yi","family":"Liu","sequence":"additional","affiliation":[{"name":"Department of Automation, Tsinghua University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yueting","family":"Chai","sequence":"additional","affiliation":[{"name":"Department of Automation, Tsinghua University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anting","family":"Zhang","sequence":"additional","affiliation":[{"name":"Department of Automation, Tsinghua University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,3]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Optimally interacting minds. Science 329, 5995","author":"Bahrami Bahador","year":"2010","unstructured":"Bahador Bahrami , Karsten Olsen , Peter\u00a0 E Latham , Andreas Roepstorff , Geraint Rees , and Chris\u00a0 D Frith . 2010. Optimally interacting minds. Science 329, 5995 ( 2010 ), 1081\u20131085. Bahador Bahrami, Karsten Olsen, Peter\u00a0E Latham, Andreas Roepstorff, Geraint Rees, and Chris\u00a0D Frith. 2010. Optimally interacting minds. Science 329, 5995 (2010), 1081\u20131085."},{"key":"e_1_3_2_1_2_1","volume-title":"Group decision making under conditions of distributed knowledge: The information asymmetries model","author":"Brodbeck C","year":"2007","unstructured":"Felix\u00a0 C Brodbeck , Rudolf Kerschreiter , Andreas Mojzisch , and Stefan Schulz-Hardt . 2007. Group decision making under conditions of distributed knowledge: The information asymmetries model . Academy of Management Review 32, 2 ( 2007 ), 459\u2013479. Felix\u00a0C Brodbeck, Rudolf Kerschreiter, Andreas Mojzisch, and Stefan Schulz-Hardt. 2007. Group decision making under conditions of distributed knowledge: The information asymmetries model.Academy of Management Review 32, 2 (2007), 459\u2013479."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.2307\/2346806"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.5555\/2892753.2892804"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1002\/bdm.1843"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.1221403"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.5555\/2986459.2986677"},{"key":"e_1_3_2_1_8_1","volume-title":"When are two heads better than one and why?Science 336, 6079","author":"Koriat Asher","year":"2012","unstructured":"Asher Koriat . 2012. When are two heads better than one and why?Science 336, 6079 ( 2012 ), 360\u2013362. Asher Koriat. 2012. When are two heads better than one and why?Science 336, 6079 (2012), 360\u2013362."},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"crossref","unstructured":"James\u00a0R Larson\u00a0Jr. 2013. In search of synergy in small group performance. Psychology Press.  James\u00a0R Larson\u00a0Jr. 2013. In search of synergy in small group performance. Psychology Press.","DOI":"10.4324\/9780203848784"},{"key":"e_1_3_2_1_10_1","volume-title":"GPM: A Generic Probabilistic Model to Recover Annotator\u2019s Behavior and Ground Truth Labeling. arXiv preprint arXiv:2003.00475(2020).","author":"Li Jing","year":"2020","unstructured":"Jing Li , Suiyi Ling , Junle Wang , Zhi Li , and Patrick\u00a0Le Callet . 2020 . GPM: A Generic Probabilistic Model to Recover Annotator\u2019s Behavior and Ground Truth Labeling. arXiv preprint arXiv:2003.00475(2020). Jing Li, Suiyi Ling, Junle Wang, Zhi Li, and Patrick\u00a0Le Callet. 2020. GPM: A Generic Probabilistic Model to Recover Annotator\u2019s Behavior and Ground Truth Labeling. arXiv preprint arXiv:2003.00475(2020)."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/DCC.2017.26"},{"key":"e_1_3_2_1_12_1","volume-title":"Proceedings of the 25th International Conference on Neural Information Processing Systems -","volume":"1","author":"Liu Qiang","year":"2012","unstructured":"Qiang Liu , Jian Peng , and Alexander Ihler . 2012 . Variational Inference for Crowdsourcing . In Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1 (Lake Tahoe, Nevada) (NIPS\u201912). Curran Associates Inc., Red Hook, NY, USA, 692\u2013700. Qiang Liu, Jian Peng, and Alexander Ihler. 2012. Variational Inference for Crowdsourcing. In Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1 (Lake Tahoe, Nevada) (NIPS\u201912). Curran Associates Inc., Red Hook, NY, USA, 692\u2013700."},{"key":"e_1_3_2_1_13_1","volume-title":"Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol.\u00a04.","author":"McDonnell Tyler","year":"2016","unstructured":"Tyler McDonnell , Matthew Lease , Mucahid Kutlu , and Tamer Elsayed . 2016 . Why is that relevant? collecting annotator rationales for relevance judgments . In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol.\u00a04. Tyler McDonnell, Matthew Lease, Mucahid Kutlu, and Tamer Elsayed. 2016. Why is that relevant? collecting annotator rationales for relevance judgments. In Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol.\u00a04."},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41562-017-0273-4"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"crossref","unstructured":"Vikas\u00a0C Raykar and Shipeng Yu. 2011. Ranking annotators for crowdsourced labeling tasks. In Advances in neural information processing systems. 1809\u20131817.  Vikas\u00a0C Raykar and Shipeng Yu. 2011. Ranking annotators for crowdsourced labeling tasks. In Advances in neural information processing systems. 1809\u20131817.","DOI":"10.1109\/NCVPRIPG.2011.14"},{"key":"e_1_3_2_1_16_1","volume-title":"Proceedings of the 2008 conference on empirical methods in natural language processing. 254\u2013263","author":"Snow Rion","year":"2008","unstructured":"Rion Snow , Brendan O\u2019connor , Dan Jurafsky , and Andrew\u00a0 Y Ng . 2008 . Cheap and fast\u2013but is it good? evaluating non-expert annotations for natural language tasks . In Proceedings of the 2008 conference on empirical methods in natural language processing. 254\u2013263 . Rion Snow, Brendan O\u2019connor, Dan Jurafsky, and Andrew\u00a0Y Ng. 2008. Cheap and fast\u2013but is it good? evaluating non-expert annotations for natural language tasks. In Proceedings of the 2008 conference on empirical methods in natural language processing. 254\u2013263."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313554"},{"key":"e_1_3_2_1_18_1","unstructured":"Tian Tian and Jun Zhu. 2015. Max-Margin Majority Voting for Learning from Crowds.. In Nips. 1621\u20131629.  Tian Tian and Jun Zhu. 2015. Max-Margin Majority Voting for Learning from Crowds.. In Nips. 1621\u20131629."},{"key":"e_1_3_2_1_19_1","volume-title":"The multidimensional wisdom of crowds. Advances in neural information processing systems 23","author":"Welinder Peter","year":"2010","unstructured":"Peter Welinder , Steve Branson , Pietro Perona , and Serge Belongie . 2010. The multidimensional wisdom of crowds. Advances in neural information processing systems 23 ( 2010 ), 2424\u20132432. Peter Welinder, Steve Branson, Pietro Perona, and Serge Belongie. 2010. The multidimensional wisdom of crowds. Advances in neural information processing systems 23 (2010), 2424\u20132432."},{"key":"e_1_3_2_1_20_1","volume-title":"Whose vote should count more: Optimal integration of labels from labelers of unknown expertise. Advances in neural information processing systems 22","author":"Whitehill Jacob","year":"2009","unstructured":"Jacob Whitehill , Ting-fan Wu, Jacob Bergsma , Javier Movellan , and Paul Ruvolo . 2009. Whose vote should count more: Optimal integration of labels from labelers of unknown expertise. Advances in neural information processing systems 22 ( 2009 ), 2035\u20132043. Jacob Whitehill, Ting-fan Wu, Jacob Bergsma, Javier Movellan, and Paul Ruvolo. 2009. Whose vote should count more: Optimal integration of labels from labelers of unknown expertise. Advances in neural information processing systems 22 (2009), 2035\u20132043."},{"key":"e_1_3_2_1_21_1","volume-title":"Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. JMLR Workshop and Conference Proceedings, 932\u2013939","author":"Yan Yan","year":"2010","unstructured":"Yan Yan , R\u00f3mer Rosales , Glenn Fung , Mark Schmidt , Gerardo Hermosillo , Luca Bogoni , Linda Moy , and Jennifer Dy . 2010 . Modeling annotator expertise: Learning when everybody knows a bit of something . In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. JMLR Workshop and Conference Proceedings, 932\u2013939 . Yan Yan, R\u00f3mer Rosales, Glenn Fung, Mark Schmidt, Gerardo Hermosillo, Luca Bogoni, Linda Moy, and Jennifer Dy. 2010. Modeling annotator expertise: Learning when everybody knows a bit of something. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. JMLR Workshop and Conference Proceedings, 932\u2013939."},{"key":"e_1_3_2_1_22_1","volume-title":"The conference of the North American chapter of the association for computational linguistics; proceedings of the main conference. 260\u2013267","author":"Zaidan Omar","year":"2007","unstructured":"Omar Zaidan , Jason Eisner , and Christine Piatko . 2007 . Using \u201cannotator rationales\u201d to improve machine learning for text categorization. In Human language technologies 2007 : The conference of the North American chapter of the association for computational linguistics; proceedings of the main conference. 260\u2013267 . Omar Zaidan, Jason Eisner, and Christine Piatko. 2007. Using \u201cannotator rationales\u201d to improve machine learning for text categorization. In Human language technologies 2007: The conference of the North American chapter of the association for computational linguistics; proceedings of the main conference. 260\u2013267."},{"key":"e_1_3_2_1_23_1","unstructured":"Denny Zhou John\u00a0C Platt Sumit Basu and Yi Mao. 2012. Learning from the wisdom of crowds by minimax entropy. (2012).  Denny Zhou John\u00a0C Platt Sumit Basu and Yi Mao. 2012. Learning from the wisdom of crowds by minimax entropy. (2012)."}],"event":{"name":"ICCSE '21: 5th International Conference on Crowd Science and Engineering","acronym":"ICCSE '21","location":"Jinan China"},"container-title":["5th International Conference on Crowd Science and Engineering"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3503181.3503184","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3503181.3503184","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:00:48Z","timestamp":1750186848000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3503181.3503184"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,16]]},"references-count":23,"alternative-id":["10.1145\/3503181.3503184","10.1145\/3503181"],"URL":"https:\/\/doi.org\/10.1145\/3503181.3503184","relation":{},"subject":[],"published":{"date-parts":[[2021,10,16]]},"assertion":[{"value":"2022-03-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}