{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,22]],"date-time":"2026-06-22T13:15:30Z","timestamp":1782134130018,"version":"3.54.5"},"reference-count":44,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2025,2,20]],"date-time":"2025-02-20T00:00:00Z","timestamp":1740009600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62276241, 62406294"],"award-info":[{"award-number":["62276241, 62406294"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Postdoctoral Fellowship Program of CPSF","award":["GZC20241546"],"award-info":[{"award-number":["GZC20241546"]}]},{"name":"Program for Science and Technology in Henan Province of China","award":["242102320055"],"award-info":[{"award-number":["242102320055"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Knowl. Discov. Data"],"published-print":{"date-parts":[[2025,4,30]]},"abstract":"<jats:p>Crowdsourcing services provide a fast and cheap way to obtain substantial labeled data by employing crowd workers on the Internet. In crowdsourcing learning, two-stage methods have been widely used, which first infer the integrated label for each instance and then build a learning model using instances with their integrated labels. However, existing two-stage methods mainly focus on how to infer more accurate integrated labels, after that, most of them directly regard the integrated labels as class labels to build a learning model, which loses the detailed worker labeling information in multiple noisy labels and thus results in sub-optimal model accuracy. To solve this problem, in this study, we take the multiple noisy labels of each instance as its attribute value vector to construct another view in addition to the original attribute view, and propose a novel two-stage method called dual-view learning from crowds (DVLFC). In DVLFC, we first pick out workers with sufficient number of labels and augment the multiple noisy label set for each instance, then we build a supervised learning model in each view and at last we fuse their class-membership probabilities to get the final classification result. Extensive experiments on both real-world and artificial crowdsourced datasets prove the effectiveness of DVLFC.<\/jats:p>","DOI":"10.1145\/3712605","type":"journal-article","created":{"date-parts":[[2025,1,17]],"date-time":"2025-01-17T13:20:44Z","timestamp":1737120044000},"page":"1-21","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Dual-View Learning from Crowds"],"prefix":"10.1145","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9914-6602","authenticated-orcid":false,"given":"Huan","family":"Zhang","sequence":"first","affiliation":[{"name":"China University of Geosciences, Wuhan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2201-3526","authenticated-orcid":false,"given":"Liangxiao","family":"Jiang","sequence":"additional","affiliation":[{"name":"China University of Geosciences, Wuhan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7269-0376","authenticated-orcid":false,"given":"Wenjun","family":"Zhang","sequence":"additional","affiliation":[{"name":"China University of Geosciences, Wuhan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9963-5169","authenticated-orcid":false,"given":"Geoffrey I.","family":"Webb","sequence":"additional","affiliation":[{"name":"Monash University, Melbourne, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,2,20]]},"reference":[{"key":"e_1_3_1_2_2","first-page":"82","volume-title":"Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence","author":"Bi Wei","year":"2014","unstructured":"Wei Bi, Liwei Wang, James T. Kwok, and Zhuowen Tu. 2014. Learning to predict from crowdsourced data. In Proceedings of the 30th Conference on Uncertainty in Artificial Intelligence, 82\u201391."},{"key":"e_1_3_1_3_2","first-page":"5304","volume-title":"Proceedings of the 36th AAAI Conference on Artificial Intelligence","author":"Chen Pengpeng","year":"2022","unstructured":"Pengpeng Chen, Hailong Sun, Yongqiang Yang, and Zhijun Chen. 2022. Adversarial learning from crowds. In Proceedings of the 36th AAAI Conference on Artificial Intelligence, 5304\u20135312."},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSC.2024.3404353"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3447548.3467409"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.1967.1053964"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.2307\/2346806"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-017-5677-x"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2021.11.021"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-018-5766-5"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2021.3082496"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2018.2836440"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2018.11.032"},{"key":"e_1_3_1_14_2","first-page":"73","volume-title":"Proceedings of the 26th AAAI Conference on Artificial Intelligence","author":"Kajino Hiroshi","year":"2012","unstructured":"Hiroshi Kajino, Yuta Tsuboi, and Hisashi Kashima. 2012. A convex formulation for learning from crowds. In Proceedings of the 26th AAAI Conference on Artificial Intelligence, 73\u201379."},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.5555\/1867135.1867170"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.engappai.2019.04.004"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2016.06.003"},{"key":"e_1_3_1_18_2","unstructured":"Hongwei Li and Bin Yu. 2014. Error rate bounds and iterative weighted majority voting for crowdsourcing. arXiv:1411.4086. Retrieved from https:\/\/arxiv.org\/pdf\/1411.4086"},{"issue":"3","key":"e_1_3_1_19_2","doi-asserted-by":"crossref","first-page":"130104","DOI":"10.1007\/s11432-020-3118-7","article-title":"Crowdsourcing aggregation with deep Bayesian learning","volume":"64","author":"Li Shao-Yuan","year":"2021","unstructured":"Shao-Yuan Li, Sheng-Jun Huang, and Songcan Chen. 2021. Crowdsourcing aggregation with deep Bayesian learning. Sci. China Inf. Sci. 64, 3 (2021), 130104.","journal-title":"Sci. China Inf. Sci."},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2018.2857766"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/2897350.2897352"},{"key":"e_1_3_1_22_2","first-page":"701","volume-title":"Proceedings of the 26th Annual Conference on Neural Information Processing Systems","author":"Liu Qiang","year":"2012","unstructured":"Qiang Liu, Jian Peng, and Alexander T. Ihler. 2012. Variational inference for crowdsourcing. In Proceedings of the 26th Annual Conference on Neural Information Processing Systems, 701\u2013709."},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.5555\/1756006.1859894"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2648786"},{"key":"e_1_3_1_25_2","first-page":"1611","volume-title":"Proceedings of the 32nd AAAI Conference on Artificial Intelligence","author":"Rodrigues Filipe","year":"2018","unstructured":"Filipe Rodrigues and Francisco C. Pereira. 2018. Deep learning from crowds. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, 1611\u20131618."},{"key":"e_1_3_1_26_2","first-page":"9837","volume-title":"Proceedings of the 33th AAAI Conference on Artificial Intelligence","author":"Sheng Victor S.","year":"2019","unstructured":"Victor S. Sheng and Jing Zhang. 2019. Machine learning with crowdsourcing: A brief summary of the past research and future directions. In Proceedings of the 33th AAAI Conference on Artificial Intelligence, 9837\u20139843."},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2017.2659740"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-020-01475-y"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.engappai.2021.104474"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2018.2860987"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2020.08.117"},{"key":"e_1_3_1_32_2","first-page":"932","volume-title":"Proceedings of the 13th International Conference on Artificial Intelligence and Statistics","author":"Yan Yan","year":"2010","unstructured":"Yan Yan, R\u00f3mer Rosales, Glenn Fung, Mark Schmidt, Gerardo Hermosillo Valadez, Luca Bogoni, Linda Moy, and Jennifer G. Dy. 2010. Modeling annotator expertise: Learning when everybody knows a bit of something. In Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, 932\u2013939."},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2019.08.071"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2022.3177634"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/JAS.2022.105434"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.5555\/2789272.2912090"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2015.2504974"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2018.2860992"},{"issue":"2","key":"e_1_3_1_39_2","first-page":"1650003:1","article-title":"A new feature selection approach to naive Bayes text classifiers","volume":"30","author":"Zhang Lungan","year":"2016","unstructured":"Lungan Zhang, Liangxiao Jiang, and Chaoqun Li. 2016. A new feature selection approach to naive Bayes text classifiers. Int. J. Pattern Recognit. Artif. Intell. 30, 2 (2016), 1650003:1\u20131650003:17.","journal-title":"Int. J. Pattern Recognit. Artif. Intell."},{"key":"e_1_3_1_40_2","first-page":"102:1","article-title":"Spectral methods meet EM: A provably optimal algorithm for crowdsourcing","volume":"17","author":"Zhang Yuchen","year":"2016","unstructured":"Yuchen Zhang, Xi Chen, Dengyong Zhou, and Michael I. Jordan. 2016. Spectral methods meet EM: A provably optimal algorithm for crowdsourcing. J. Mach. Learn. Res. 17 (2016), 102:1\u2013102:44.","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2017.2738643"},{"key":"e_1_3_1_42_2","first-page":"2204","volume-title":"Proceedings of the 26th Annual Conference on Neural Information Processing Systems","author":"Zhou Dengyong","year":"2012","unstructured":"Dengyong Zhou, John C. Platt, Sumit Basu, and Yi Mao. 2012. Learning from the wisdom of crowds by minimax entropy. In Proceedings of the 26th Annual Conference on Neural Information Processing Systems, 2204\u20132212."},{"key":"e_1_3_1_43_2","first-page":"685","volume-title":"Proceedings of the 2017 IEEE International Conference on Data Mining.","author":"Zhou Yao","year":"2017","unstructured":"Yao Zhou and Jingrui He. 2017. A randomized approach for crowdsourcing in the presence of multiple views. In Proceedings of the 2017 IEEE International Conference on Data Mining. IEEE Computer Society, 685\u2013694."},{"key":"e_1_3_1_44_2","doi-asserted-by":"crossref","first-page":"3231","DOI":"10.1145\/3292500.3332277","volume-title":"Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining","author":"Zhou Yao","year":"2019","unstructured":"Yao Zhou, Fenglong Ma, Jing Gao, and Jingrui He. 2019. Optimizing the wisdom of the crowd: Inference, learning, and teaching. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, 3231\u20133232."},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1137\/1.9781611974973.65"}],"container-title":["ACM Transactions on Knowledge Discovery from Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3712605","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3712605","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:18:37Z","timestamp":1750295917000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3712605"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,2,20]]},"references-count":44,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2025,4,30]]}},"alternative-id":["10.1145\/3712605"],"URL":"https:\/\/doi.org\/10.1145\/3712605","relation":{},"ISSN":["1556-4681","1556-472X"],"issn-type":[{"value":"1556-4681","type":"print"},{"value":"1556-472X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,2,20]]},"assertion":[{"value":"2024-03-20","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-12-21","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-02-20","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}