{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,16]],"date-time":"2026-04-16T15:34:31Z","timestamp":1776353671235,"version":"3.51.2"},"reference-count":50,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2021,10,22]],"date-time":"2021-10-22T00:00:00Z","timestamp":1634860800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Knowl. Discov. Data"],"published-print":{"date-parts":[[2022,6,30]]},"abstract":"<jats:p>In real-world tasks, obtaining a large set of noise-free data can be prohibitively expensive. Therefore, recent research tries to enable machine learning to work with weakly supervised datasets, such as inaccurate or incomplete data. However, the previous literature treats each type of weak supervision individually, although, in most cases, different types of weak supervision tend to occur simultaneously. Therefore, in this article, we present Smart MEnDR, a Classification Model that applies Ensemble Learning and Data-driven Rectification to deal with inaccurate and incomplete supervised datasets. The model first applies a preliminary phase of ensemble learning in which the noisy data points are detected while exploiting the unlabelled data. The phase employs a semi-supervised technique with maximum likelihood estimation to decide on the disagreement rate. Second, the proposed approach applies an iterative meta-learning step to tackle the problem of knowing which points should be made correct to improve the performance of the final classifier. To evaluate the proposed framework, we report the classification performance, noise detection, and the labelling accuracy of the proposed method against state-of-the-art techniques. The experimental results demonstrate the effectiveness of the proposed framework in detecting noise, providing correct labels, and attaining high classification performance.<\/jats:p>","DOI":"10.1145\/3473910","type":"journal-article","created":{"date-parts":[[2021,10,23]],"date-time":"2021-10-23T04:28:40Z","timestamp":1634963320000},"page":"1-33","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Semi-Supervised Ensemble Learning for Dealing with Inaccurate and Incomplete Supervision"],"prefix":"10.1145","volume":"16","author":[{"given":"Mona","family":"Nashaat","sequence":"first","affiliation":[{"name":"Electrical and Computer Engineering, University of Alberta, Edmonton, Alberta, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Aindrila","family":"Ghosh","sequence":"additional","affiliation":[{"name":"Electrical and Computer Engineering, University of Alberta, Edmonton, Alberta, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"James","family":"Miller","sequence":"additional","affiliation":[{"name":"Electrical and Computer Engineering, University of Alberta, Edmonton, Alberta, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shaikh","family":"Quader","sequence":"additional","affiliation":[{"name":"IBM Canada Software Lab, IBM Canada, Toronto, Ontario, Canada"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,10,22]]},"reference":[{"issue":"1","key":"e_1_3_2_2_2","first-page":"334","article-title":"Towards safe weakly supervised learning","volume":"43","author":"Li Y. F.","year":"2019","journal-title":"IEEE TransActions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_3_2_3_2","first-page":"273","volume-title":"Proceedings of the 34th International Conference on Machine Learning","volume":"70","author":"Bach S. H.","year":"2017"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2018.2849394"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2017.10.026"},{"key":"e_1_3_2_6_2","first-page":"3581","volume-title":"Proceedings of the 27th International Conference on Advances in Neural Information Processing Systems","author":"Kingma D. P.","year":"2014"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-012-0507-8"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-58628-1_43"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2017.11.012"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-016-0475-9"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2807779"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10247-4_5"},{"key":"e_1_3_2_13_2","first-page":"3235","volume-title":"Proceedings of the 32nd International Conference on Advances in Neural Information Processing Systems","author":"Oliver A.","year":"2018"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.5555\/3294771.3294794"},{"key":"e_1_3_2_15_2","unstructured":"L.-Z. Guo F. Kuang Z.-X. Liu Y.-F. Li N. Ma and X.-H. Qie. Weakly supervised learning meets ride-sharing user experience enhancement. arXiv:2001.09027. Retrieved from https:\/\/arxiv.org\/abs\/2001.09027."},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330902"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-05090-0_5"},{"key":"e_1_3_2_18_2","first-page":"265","volume-title":"Frontiers in Artificial Intelligence and Applications","author":"Saman R.","year":"2019"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2015.04.002"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2013.2292894"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2016.05.035"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2015.2475750"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2018.2816984"},{"key":"e_1_3_2_24_2","unstructured":"X. Liu D. Zachariah J. W\u00e5gberg and T. B. Sch\u00f6n. 2019. Reliable semi-supervised learning when labels are missing at random. arXiv:1811.10947. Retrieved from https:\/\/arxiv.org\/abs1811.10947."},{"key":"e_1_3_2_25_2","first-page":"1636","volume-title":"Proceedings of the 34th International Conference on Machine Learning Research","author":"Jain V.","year":"2017"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCYB.2018.2804326"},{"key":"e_1_3_2_27_2","first-page":"3","volume-title":"Computer Communications","author":"Ding Y.","year":"2016"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1093\/nsr\/nwx106"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-016-0469-7"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1007\/s13042-017-0645-0"},{"key":"e_1_3_2_31_2","volume-title":"Proceedings of the 6th AAAI Conference on Human Computation and Crowdsourcing","author":"Lin C. H.","year":"2018"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-014-0781-x"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-018-1244-4"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2014.12.086"},{"key":"e_1_3_2_35_2","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 34","author":"Shang Y.","year":"2020"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2019.2922396"},{"key":"e_1_3_2_37_2","first-page":"131","volume-title":"Information and Software Technology","author":"Nashaat M.","year":"2019"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/BigData.2018.8622459"},{"issue":"1","key":"e_1_3_2_39_2","first-page":"1","article-title":"Statistical comparisons of classifiers over multiple data sets","volume":"7","author":"Dem\u0161ar J.","year":"2006","journal-title":"Machine Learning Research"},{"key":"e_1_3_2_40_2","unstructured":"W. Gao B. B. Yang and Z. H. Zhou. 2018. On the resistance of nearest neighbor to random noisy labels. arXiv:1607.07526. Retrieved from http:\/\/arxiv.org\/abs\/1607.07526."},{"key":"e_1_3_2_41_2","first-page":"119","volume-title":"Pattern Recognition","author":"Sabzevari M.","year":"2018"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-016-9518-2"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/JBHI.2020.2974425"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-018-1085-3"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2017.07.012"},{"key":"e_1_3_2_46_2","first-page":"11174","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Lin Y."},{"key":"e_1_3_2_47_2","first-page":"1134","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Yang M."},{"key":"e_1_3_2_48_2","doi-asserted-by":"crossref","unstructured":"B. Liu K. Blekas and G. Tsoumakas. 2021. Multi-Label Sampling based on Local Label Imbalance. Pattern Recognition (In press).","DOI":"10.1016\/j.patcog.2021.108294"},{"key":"e_1_3_2_49_2","volume-title":"IEEE Transactions on Cybernetics","author":"Zhang J.","year":"2020"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.engappai.2019.04.004"},{"key":"e_1_3_2_51_2","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","volume":"33","author":"Huang Z.","year":"2020"}],"container-title":["ACM Transactions on Knowledge Discovery from Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3473910","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3473910","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:48:46Z","timestamp":1750193326000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3473910"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,22]]},"references-count":50,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2022,6,30]]}},"alternative-id":["10.1145\/3473910"],"URL":"https:\/\/doi.org\/10.1145\/3473910","relation":{},"ISSN":["1556-4681","1556-472X"],"issn-type":[{"value":"1556-4681","type":"print"},{"value":"1556-472X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,22]]},"assertion":[{"value":"2020-05-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-07-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-10-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}