{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,3]],"date-time":"2025-09-03T10:34:01Z","timestamp":1756895641912,"version":"3.41.0"},"reference-count":56,"publisher":"Association for Computing Machinery (ACM)","issue":"10","license":[{"start":{"date-parts":[[2024,10,30]],"date-time":"2024-10-30T00:00:00Z","timestamp":1730246400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100018537","name":"National Science and Technology Major Project","doi-asserted-by":"crossref","award":["2022ZD0117802"],"award-info":[{"award-number":["2022ZD0117802"]}],"id":[{"id":"10.13039\/501100018537","id-type":"DOI","asserted-by":"crossref"}]},{"name":"General Program of National Natural Science Foundation of China","award":["62372403"],"award-info":[{"award-number":["62372403"]}]},{"name":"Earth System Big Data Platform of the School of Earth Sciences, Zhejiang University, and the Fundamental Research Funds for the Central Universities","award":["226-2022-00051"],"award-info":[{"award-number":["226-2022-00051"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2024,10,31]]},"abstract":"<jats:p>We focus on the challenging problem of learning an unbiased classifier from a large number of potentially relevant but noisily labeled web images given only a few clean labeled images. This problem is particularly practical because it reduces the expensive annotation costs by utilizing freely accessible web images with noisy labels. Typically, prototypes are representative images or features used to classify or identify other images. However, in the few clean and many noisy scenarios, the class prototype can be severely biased due to the presence of irrelevant noisy images. The resulting prototypes are less compact and discriminative, as previous methods do not take into account the diverse range of images in the noisy web image collections. On the other hand, the relation modeling between noisy and clean images is not learned for the class prototype generation in an end-to-end manner, which results in a suboptimal class prototype. In this article, we introduce a similarity maximization loss named SimNoiPro. Our SimNoiPro first generates noise-tolerant hybrid prototypes composed of clean and noise-tolerant prototypes and then pulls them closer to each other. Our approach considers the diversity of noisy images by explicit division and overcomes the optimization discrepancy issue. This enables better relation modeling between clean and noisy images and helps extract judicious information from the noisy image set. The evaluation results on two extended few-shot classification benchmarks confirm that our SimNoiPro outperforms prior methods in measuring image relations and cleaning noisy data.<\/jats:p>","DOI":"10.1145\/3672396","type":"journal-article","created":{"date-parts":[[2024,7,8]],"date-time":"2024-07-08T13:29:16Z","timestamp":1720445356000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Noise-Tolerant Hybrid Prototypical Learning with Noisy Web Data"],"prefix":"10.1145","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7634-2507","authenticated-orcid":false,"given":"Chao","family":"Liang","sequence":"first","affiliation":[{"name":"Zhejiang University, Zhejiang, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4093-7557","authenticated-orcid":false,"given":"Linchao","family":"Zhu","sequence":"additional","affiliation":[{"name":"Zhejiang University, Zhejiang, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8783-8313","authenticated-orcid":false,"given":"Zongxin","family":"Yang","sequence":"additional","affiliation":[{"name":"Zhejiang University, Zhejiang, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8365-4741","authenticated-orcid":false,"given":"Wei","family":"Chen","sequence":"additional","affiliation":[{"name":"Zhejiang University, Zhejiang, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0512-880X","authenticated-orcid":false,"given":"Yi","family":"Yang","sequence":"additional","affiliation":[{"name":"Zhejiang University, Zhejiang, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,10,30]]},"reference":[{"key":"e_1_3_1_2_2","first-page":"312","volume-title":"Proceedings of the International Conference on Machine Learning (ICML)","author":"Arazo Eric","year":"2019","unstructured":"Eric Arazo, Diego Ortego, Paul Albert, Noel O\u2019Connor, and Kevin McGuinness. 2019. Unsupervised label noise modeling and loss correction. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 312\u2013321."},{"key":"e_1_3_1_3_2","first-page":"9912","volume-title":"Proceedings of the Advances in Neural Information Processing Systems (NIPS)","author":"Caron Mathilde","year":"2020","unstructured":"Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. 2020. Unsupervised learning of visual features by contrasting cluster assignments. In Proceedings of the Advances in Neural Information Processing Systems (NIPS), Vol. 33. Curran Associates, Inc., 9912\u20139924."},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00353"},{"key":"e_1_3_1_5_2","first-page":"30284","volume-title":"Proceedings of the Advances in Neural Information Processing Systems (NIPS)","volume":"34","author":"Englesson Erik","year":"2021","unstructured":"Erik Englesson and Hossein Azizpour. 2021. Generalized Jensen-Shannon divergence loss for learning with noisy labels. In Proceedings of the Advances in Neural Information Processing Systems (NIPS), Vol. 34. 30284\u201330297."},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3390891"},{"key":"e_1_3_1_7_2","first-page":"1126","volume-title":"Proceedings of the International Conference on Machine Learning (ICML)","author":"Finn Chelsea","year":"2017","unstructured":"Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 1126\u20131135."},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00459"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.328"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3511917"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394171.3413811"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.322"},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58607-2_17"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.00463"},{"key":"e_1_3_1_16_2","first-page":"2304","volume-title":"Proceedings of the International Conference on Machine Learning (ICML)","author":"Jiang Lu","year":"2018","unstructured":"Lu Jiang, Zhengyuan Zhou, Thomas Leung, Li-Jia Li, and Li Fei-Fei. 2018. Mentornet: Learning data-driven curriculum for very deep neural networks on corrupted labels. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 2304\u20132313."},{"key":"e_1_3_1_17_2","first-page":"1097","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","volume":"25","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 25. 1097\u20131105."},{"key":"e_1_3_1_18_2","volume-title":"Proceedings of the International Conference on Learning Representations (ICLR)","author":"Li Junnan","year":"2021","unstructured":"Junnan Li, Caiming Xiong, and Steven Hoi. 2021a. MoPro: Webly supervised learning with momentum prototypes. In Proceedings of the International Conference on Learning Representations (ICLR)."},{"key":"e_1_3_1_19_2","volume-title":"Proceedings of the International Conference on Learning Representations (ICLR)","author":"Li Junnan","year":"2021","unstructured":"Junnan Li, Pan Zhou, Caiming Xiong, and Steven Hoi. 2021b. Prototypical contrastive learning of unsupervised representations. In Proceedings of the International Conference on Learning Representations (ICLR)."},{"key":"e_1_3_1_20_2","volume-title":"Proceedings of the International Conference on Machine Learning (ICML)","author":"Li Wei","year":"2024","unstructured":"Wei Li, Hehe Fan, Yongkang Wong, Yi Yang, and Mohan Kankanhalli. 2024. Improve context understanding in multimodal large language models via multimodal composition learning. In Proceedings of the International Conference on Machine Learning (ICML)."},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2023.3270103"},{"key":"e_1_3_1_22_2","unstructured":"Chao Liang Linchao Zhu Humphrey Shi and Yi Yang. 2023b. Combating label noise with a general surrogate model for sample selection. arXiv:2310.10463. Retrieved from https:\/\/arxiv.org\/abs\/2310.10463"},{"key":"e_1_3_1_23_2","unstructured":"Ilya Loshchilov and Frank Hutter. 2016. SGDR: Stochastic gradient descent with warm restarts. arXiv:1608.03983. Retrieved from https:\/\/arxiv.org\/abs\/1608.03983"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2015.2495248"},{"key":"e_1_3_1_25_2","first-page":"3355","volume-title":"Proceedings of the International Conference on Machine Learning (ICML)","author":"Ma Xingjun","year":"2018","unstructured":"Xingjun Ma, Yisen Wang, Michael E Houle, Shuo Zhou, Sarah Erfani, Shutao Xia, Sudanthi Wijewickrema, and James Bailey. 2018. Dimensionality-driven learning with noisy labels. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 3355\u20133364."},{"key":"e_1_3_1_26_2","volume-title":"International Conference on Learning Representations (ICLR)","author":"Ravi Sachin","year":"2017","unstructured":"Sachin Ravi and Hugo Larochelle. 2017. Optimization as a model for few-shot learning. In International Conference on Learning Representations (ICLR)."},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.91"},{"key":"e_1_3_1_28_2","unstructured":"Scott Reed Honglak Lee Dragomir Anguelov Christian Szegedy Dumitru Erhan and Andrew Rabinovich. 2014. Training deep neural networks on noisy labels with bootstrapping. arXiv:1412.6596. Retrieved from https:\/\/arxiv.org\/abs\/1412.6596"},{"key":"e_1_3_1_29_2","first-page":"4334","volume-title":"Proceedings of the International Conference on Machine Learning (ICML)","author":"Ren Mengye","year":"2018","unstructured":"Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. 2018. Learning to reweight examples for robust deep learning. In Proceedings of the International Conference on Machine Learning (ICML). PMLR, 4334\u20134343."},{"key":"e_1_3_1_30_2","volume-title":"Proceedings of the Advances in Neural Information Processing Systems (NIPS)","author":"Ren Shaoqing","year":"2015","unstructured":"Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster R-CNN: Towards real-time object detection with region proposal networks. In Proceedings of the Advances in Neural Information Processing Systems (NIPS), Vol. 28. Curran Associates, Inc."},{"key":"e_1_3_1_31_2","doi-asserted-by":"crossref","unstructured":"Simone Ricci Tiberio Uricchio and Alberto Del Bimbo. 2023. Meta-learning advisor networks for long-tail and noisy labels in social image classification. ACM Transactions on Multimedia Computing Communications and Applications 19 5s (Jun 2023) Article 169 23 pages.","DOI":"10.1145\/3584360"},{"key":"e_1_3_1_32_2","unstructured":"Andrei A. Rusu Dushyant Rao Jakub Sygnowski Oriol Vinyals Razvan Pascanu Simon Osindero and Raia Hadsell. 2018. Meta-learning with latent embedding optimization. arXiv:1807.05960. Retrieved from https:\/\/arxiv.org\/abs\/1807.05960"},{"key":"e_1_3_1_33_2","unstructured":"Adam Santoro Sergey Bartunov Matthew Botvinick Daan Wierstra and Timothy Lillicrap. 2016. One-shot learning with memory-augmented neural networks. arXiv:1605.06065. Retrieved from https:\/\/arxiv.org\/abs\/1605.06065"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3608478"},{"key":"e_1_3_1_35_2","unstructured":"Jun Shu Qi Xie Lixuan Yi Qian Zhao Sanping Zhou Zongben Xu and Deyu Meng. 2019. Meta-weight-net: Learning an explicit mapping for sample weighting. arXiv:1902.07379. Retrieved from https:\/\/arxiv.org\/abs\/1902.07379"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00535"},{"key":"e_1_3_1_37_2","volume-title":"Proceedings of the Advances in Neural Information Processing Systems (NIPS)","author":"Snell Jake","year":"2017","unstructured":"Jake Snell, Kevin Swersky, and Richard Zemel. 2017. Prototypical networks for few-shot learning. In Proceedings of the Advances in Neural Information Processing Systems (NIPS), Vol. 30. Curran Associates, Inc."},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2022.3158001"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00131"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00582"},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3555314"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/2812802"},{"key":"e_1_3_1_43_2","volume-title":"Advances in Neural Information Processing Systems (NIPS)","author":"Vinyals Oriol","year":"2016","unstructured":"Oriol Vinyals, Charles Blundell, Timothy Lillicrap, koray kavukcuoglu, and Daan Wierstra. 2016. Matching Networks for One Shot Learning. In Advances in Neural Information Processing Systems (NIPS), Vol. 29. Curran Associates, Inc."},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2022.3204444"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2021.3085208"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1631\/FITEE.2100463"},{"key":"e_1_3_1_47_2","volume-title":"Proceedings of the International Conference on Machine Learning (ICML)","author":"Yang Zongxin","year":"2024","unstructured":"Zongxin Yang, Guikun Chen, Xiaodi Li, Wenguan Wang, and Yi Yang. 2024. DoraemonGPT: Toward understanding dynamic scenes with large language models (exemplified as a video agent). In Proceedings of the International Conference on Machine Learning (ICML)."},{"key":"e_1_3_1_48_2","first-page":"1","article-title":"Scalable video object segmentation with identification mechanism","author":"Yang Zongxin","year":"2024","unstructured":"Zongxin Yang, Jiaxu Miao, Yunchao Wei, Wenguan Wang, Xiaohan Wang, and Yi Yang. 2024. Scalable video object segmentation with identification mechanism. IEEE Transactions on Pattern Analysis and Machine Intelligence (2024), 1\u201315.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"issue":"9","key":"e_1_3_1_49_2","first-page":"4701","article-title":"Collaborative video object segmentation by multi-scale foreground-background integration","volume":"44","author":"Yang Zongxin","year":"2021","unstructured":"Zongxin Yang, Yunchao Wei, and Yi Yang. 2021. Collaborative video object segmentation by multi-scale foreground-background integration. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 9 (2021), 4701\u20134712.","journal-title":"IEEE Transactions on Pattern Analysis and Machine Intelligence"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00718"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3472809"},{"key":"e_1_3_1_52_2","unstructured":"Chiyuan Zhang Samy Bengio Moritz Hardt Benjamin Recht and Oriol Vinyals. 2016. Understanding deep learning requires rethinking generalization. arXiv:1611.03530. Retrieved from https:\/\/arxiv.org\/abs\/1611.03530"},{"key":"e_1_3_1_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01222"},{"key":"e_1_3_1_54_2","unstructured":"Yue Zhang Hehe Fan and Yi Yang. 2024. Prompt-aware adapter: Towards learning adaptive visual tokens for multimodal large language models. arXiv:2405.15684. Retrieved from https:\/\/arxiv.org\/abs\/2405.15684"},{"key":"e_1_3_1_55_2","volume-title":"International Conference on Learning Representations (ICLR)","author":"Zhao Shuai","year":"2024","unstructured":"Shuai Zhao, Xiaohan Wang, Linchao Zhu, and Yi Yang. 2024. Test-time adaptation with CLIP reward for zero-shot generalization in vision-language models. In International Conference on Learning Representations (ICLR)."},{"key":"e_1_3_1_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2723009"},{"key":"e_1_3_1_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2021.3057503"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3672396","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3672396","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:57:49Z","timestamp":1750294669000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3672396"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,30]]},"references-count":56,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2024,10,31]]}},"alternative-id":["10.1145\/3672396"],"URL":"https:\/\/doi.org\/10.1145\/3672396","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"type":"print","value":"1551-6857"},{"type":"electronic","value":"1551-6865"}],"subject":[],"published":{"date-parts":[[2024,10,30]]},"assertion":[{"value":"2023-07-29","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-05-25","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-10-30","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}