{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,3]],"date-time":"2026-05-03T12:57:51Z","timestamp":1777813071007,"version":"3.51.4"},"reference-count":36,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2021,9,3]],"date-time":"2021-09-03T00:00:00Z","timestamp":1630627200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62006118, 62006119, 61836013, and 91746301"],"award-info":[{"award-number":["62006118, 62006119, 61836013, and 91746301"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Natural Science Foundation of Jiangsu Province of China","award":["BK20200460, BK20190444"],"award-info":[{"award-number":["BK20200460, BK20190444"]}]},{"name":"CCF-Baidu Open Fund","award":["CCF-BAIDU OF2020011"],"award-info":[{"award-number":["CCF-BAIDU OF2020011"]}]},{"name":"Baidu TIC Open Fund"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Knowl. Discov. Data"],"published-print":{"date-parts":[[2022,4,30]]},"abstract":"<jats:p>Open set classification (OSC) tackles the problem of determining whether the data are in-class or out-of-class during inference, when only provided with a set of in-class examples at training time. Traditional OSC methods usually train discriminative or generative models with the owned in-class data, and then utilize the pre-trained models to classify test data directly. However, these methods always suffer from the embedding confusion problem, i.e., partial out-of-class instances are mixed with in-class ones of similar semantics, making it difficult to classify. To solve this problem, we unify semi-supervised learning to develop a novel OSC algorithm, S2OSC, which incorporates out-of-class instances filtering and model re-training in a transductive manner. In detail, given a pool of newly coming test data, S2OSC firstly filters the mostly distinct out-of-class instances using the pre-trained model, and annotates super-class for them. Then, S2OSC trains a holistic classification model by combing in-class and out-of-class labeled data with the remaining unlabeled test data in a semi-supervised paradigm. Furthermore, considering that data are usually in the streaming form in real applications, we extend S2OSC into an incremental update framework (I-S2OSC), and adopt a knowledge memory regularization to mitigate the catastrophic forgetting problem in incremental update. Despite the simplicity of proposed models, the experimental results show that S2OSC achieves state-of-the-art performance across a variety of OSC tasks, including 85.4% of F1 on CIFAR-10 with only 300 pseudo-labels. We also demonstrate how S2OSC can be expanded to incremental OSC setting effectively with streaming data.<\/jats:p>","DOI":"10.1145\/3468675","type":"journal-article","created":{"date-parts":[[2021,9,4]],"date-time":"2021-09-04T04:05:52Z","timestamp":1630728352000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["S2OSC: A Holistic Semi-Supervised Approach for Open Set Classification"],"prefix":"10.1145","volume":"16","author":[{"given":"Yang","family":"Yang","sequence":"first","affiliation":[{"name":"Nanjing University of Science and Technology, Nanjing, Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hongchen","family":"Wei","sequence":"additional","affiliation":[{"name":"Nanjing University of Science and Technology, Nanjing, Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhen-Qiang","family":"Sun","sequence":"additional","affiliation":[{"name":"Nanjing Normal University, Nanjing, Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Guang-Yu","family":"Li","sequence":"additional","affiliation":[{"name":"Nanjing University of Science and Technology, Nanjing, Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuanchun","family":"Zhou","sequence":"additional","affiliation":[{"name":"Computer Network and Information Center, Chinese Academy of Sciences, Dongshengnanlu, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hui","family":"Xiong","sequence":"additional","affiliation":[{"name":"Rutgers University, Newark, New Jersey"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jian","family":"Yang","sequence":"additional","affiliation":[{"name":"Nanjing University of Science and Technology, Nanjing, Jiangsu, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,9,3]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Proceedings of the International Conference on Learning Representations.","author":"Berthelot David","year":"2020","unstructured":"David Berthelot , Nicholas Carlini , Ekin D. Cubuk , Alex Kurakin , Kihyuk Sohn , Han Zhang , and Colin Raffel . 2020 . ReMixMatch: Semi-supervised learning with distribution matching and augmentation anchoring . In Proceedings of the International Conference on Learning Representations. David Berthelot, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Kihyuk Sohn, Han Zhang, and Colin Raffel. 2020. ReMixMatch: Semi-supervised learning with distribution matching and augmentation anchoring. In Proceedings of the International Conference on Learning Representations."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2019.00109"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.575"},{"key":"e_1_2_1_4_1","volume-title":"Proceedings of the European Conference on Computer Vision. 556\u2013572","author":"Chaudhry Arslan","unstructured":"Arslan Chaudhry , Puneet K. Dokania , Thalaiyasingam Ajanthan , and Philip H. S. Torr . 2018. Riemannian walk for incremental learning: understanding forgetting and intransigence . In Proceedings of the European Conference on Computer Vision. 556\u2013572 . Arslan Chaudhry, Puneet K. Dokania, Thalaiyasingam Ajanthan, and Philip H. S. Torr. 2018. Riemannian walk for incremental learning: understanding forgetting and intransigence. In Proceedings of the European Conference on Computer Vision. 556\u2013572."},{"key":"e_1_2_1_5_1","volume-title":"Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 113\u2013123","author":"Cubuk Ekin D.","unstructured":"Ekin D. Cubuk , Barret Zoph , Dandelion Mane , Vijay Vasudevan , and Quoc V. Le . 2019. AutoAugment: Learning augmentation strategies from data . In Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 113\u2013123 . Ekin D. Cubuk, Barret Zoph, Dandelion Mane, Vijay Vasudevan, and Quoc V. Le. 2019. AutoAugment: Learning augmentation strategies from data. In Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 113\u2013123."},{"key":"e_1_2_1_6_1","volume-title":"Proceedings of the 28th AAAI Conference on Artificial Intelligence. 1760\u20131766","author":"Da Qing","year":"2014","unstructured":"Qing Da , Yang Yu , and Zhi-Hua Zhou . 2014 . Learning with augmented class by exploiting unlabeled data . In Proceedings of the 28th AAAI Conference on Artificial Intelligence. 1760\u20131766 . Qing Da, Yang Yu, and Zhi-Hua Zhou. 2014. Learning with augmented class by exploiting unlabeled data. In Proceedings of the 28th AAAI Conference on Artificial Intelligence. 1760\u20131766."},{"key":"e_1_2_1_7_1","volume-title":"Taylor","author":"Devries Terrance","year":"2017","unstructured":"Terrance Devries and Graham W . Taylor . 2017 . Improved regularization of convolutional neural networks with cutout. arXiv:1708.04552. Retrieved from https:\/\/arxiv.org\/abs\/1708.04552. Terrance Devries and Graham W. Taylor. 2017. Improved regularization of convolutional neural networks with cutout. arXiv:1708.04552. Retrieved from https:\/\/arxiv.org\/abs\/1708.04552."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.5244\/C.31.42"},{"key":"e_1_2_1_9_1","volume-title":"Recent Advances in Open Set Recognition: A Survey","author":"Geng Chuanxing","unstructured":"Chuanxing Geng , Sheng-Jun Huang , and Songcan Chen . 2020. Recent Advances in Open Set Recognition: A Survey . IEEE Transactions on Pattern Analysis and Machine Intelligence . early access. DOI:10.1109\/TPAMI.2020.2981604 10.1109\/TPAMI.2020.2981604 Chuanxing Geng, Sheng-Jun Huang, and Songcan Chen. 2020. Recent Advances in Open Set Recognition: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence. early access. DOI:10.1109\/TPAMI.2020.2981604"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00849"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_1_12_1","volume-title":"Proceedings of the International Conference on Learning Representations.","author":"Hendrycks Dan","year":"2017","unstructured":"Dan Hendrycks and Kevin Gimpel . 2017 . A baseline for detecting misclassified and out-of-distribution examples in neural networks . In Proceedings of the International Conference on Learning Representations. Dan Hendrycks and Kevin Gimpel. 2017. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In Proceedings of the International Conference on Learning Representations."},{"key":"e_1_2_1_13_1","unstructured":"Geoffrey E. Hinton Oriol Vinyals and Jeffrey Dean. 2015. Distilling the knowledge in a neural network. arXiv:1503.02531. Retrieved from https:\/\/arxiv.org\/abs\/1503.02531.  Geoffrey E. Hinton Oriol Vinyals and Jeffrey Dean. 2015. Distilling the knowledge in a neural network. arXiv:1503.02531. Retrieved from https:\/\/arxiv.org\/abs\/1503.02531."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2018.8461700"},{"key":"e_1_2_1_15_1","volume-title":"Proceedings of the International Conference on Learning Representations.","author":"Kim Ki Hyun","unstructured":"Ki Hyun Kim , Sangwoo Shim , Yongsub Lim , Jongseob Jeon , Jeongwoo Choi , Byungchan Kim , and Andre S. Yoon . 2020. RaPP: Novelty detection with reconstruction along projection pathway . In Proceedings of the International Conference on Learning Representations. Ki Hyun Kim, Sangwoo Shim, Yongsub Lim, Jongseob Jeon, Jeongwoo Choi, Byungchan Kim, and Andre S. Yoon. 2020. RaPP: Novelty detection with reconstruction along projection pathway. In Proceedings of the International Conference on Learning Representations."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1611835114"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1561\/2200000019"},{"key":"e_1_2_1_19_1","volume-title":"Burges","author":"LeCun Yann","year":"1998","unstructured":"Yann LeCun , Corinna Cortes , and Christopher J. C . Burges . 1998 . The MNIST database of handwritten digits, 1998. Retrieved from http:\/\/yann.lecun.com\/exdb\/mnist. 10 (1998), 34. Yann LeCun, Corinna Cortes, and Christopher J. C. Burges. 1998. The MNIST database of handwritten digits, 1998. Retrieved from http:\/\/yann.lecun.com\/exdb\/mnist. 10 (1998), 34."},{"key":"e_1_2_1_20_1","volume-title":"Proceedings of the 31st International Conference on Neural Information Processing Systems. 4655\u20134665","author":"Lee Sang-Woo","year":"2017","unstructured":"Sang-Woo Lee , Jin-Hwa Kim , Jaehyun Jun , Jung-Woo Ha , and Byoung-Tak Zhang . 2017 . Overcoming catastrophic forgetting by incremental moment matching . In Proceedings of the 31st International Conference on Neural Information Processing Systems. 4655\u20134665 . Sang-Woo Lee, Jin-Hwa Kim, Jaehyun Jun, Jung-Woo Ha, and Byoung-Tak Zhang. 2017. Overcoming catastrophic forgetting by incremental moment matching. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 4655\u20134665."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3343031.3350901"},{"key":"e_1_2_1_22_1","volume-title":"Proceedings of the International Conference on Learning Representations.","author":"Liang Shiyu","unstructured":"Shiyu Liang , Yixuan Li , and R. Srikant . 2018. Enhancing the reliability of out-of-distribution image detection in neural networks . In Proceedings of the International Conference on Learning Representations. Shiyu Liang, Yixuan Li, and R. Srikant. 2018. Enhancing the reliability of out-of-distribution image detection in neural networks. In Proceedings of the International Conference on Learning Representations."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2008.17"},{"key":"e_1_2_1_24_1","volume-title":"Proceedings of the 33rd International Conference on Machine Learning. 507\u2013516","author":"Liu Weiyang","year":"2016","unstructured":"Weiyang Liu , Yandong Wen , Zhiding Yu , and Meng Yang . 2016 . Large-margin softmax loss for convolutional neural networks . In Proceedings of the 33rd International Conference on Machine Learning. 507\u2013516 . Weiyang Liu, Yandong Wen, Zhiding Yu, and Meng Yang. 2016. Large-margin softmax loss for convolutional neural networks. In Proceedings of the 33rd International Conference on Machine Learning. 507\u2013516."},{"key":"e_1_2_1_25_1","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"Der Maaten Laurens Van","year":"2008","unstructured":"Laurens Van Der Maaten and Geoffrey Hinton . 2008 . Visualizing data using t-SNE . Journal of Machine Learning Research 9 , 86 (2008), 2579 \u2013 2605 . Laurens Van Der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9, 86 (2008), 2579\u20132605.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_2_1_26_1","volume-title":"Proceedings of the 31st AAAI Conference on Artificial Intelligence. 2373\u20132379","author":"Mu Xin","year":"2017","unstructured":"Xin Mu , Feida Zhu , Juan Du , Ee-Peng Lim , and Zhi-Hua Zhou . 2017 . Streaming classification with emerging new class by class matrix sketching . In Proceedings of the 31st AAAI Conference on Artificial Intelligence. 2373\u20132379 . Xin Mu, Feida Zhu, Juan Du, Ee-Peng Lim, and Zhi-Hua Zhou. 2017. Streaming classification with emerging new class by class matrix sketching. In Proceedings of the 31st AAAI Conference on Artificial Intelligence. 2373\u20132379."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01231-1_38"},{"key":"e_1_2_1_28_1","volume-title":"Proceedings of the 25st International Conference on Neural Information Processing Systems. 1\u20139.","author":"Netzer Yuval","unstructured":"Yuval Netzer , Tao Wang , Adam Coates , Alessandro Bissacco , Bo Wu , and Andrew Y. Ng . 2011. Reading digits in natural images with unsupervised feature learning . In Proceedings of the 25st International Conference on Neural Information Processing Systems. 1\u20139. Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. 2011. Reading digits in natural images with unsupervised feature learning. In Proceedings of the 25st International Conference on Neural Information Processing Systems. 1\u20139."},{"key":"e_1_2_1_29_1","volume-title":"Connectionist models of recognition memory: constraints imposed by learning and forgetting functions.Psychological Review 97, 2","author":"Ratcliff Roger","year":"1990","unstructured":"Roger Ratcliff . 1990. Connectionist models of recognition memory: constraints imposed by learning and forgetting functions.Psychological Review 97, 2 ( 1990 ), 285\u2013308. Roger Ratcliff. 1990. Connectionist models of recognition memory: constraints imposed by learning and forgetting functions.Psychological Review 97, 2 (1990), 285\u2013308."},{"key":"e_1_2_1_30_1","volume-title":"Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. 5533\u20135542","author":"Rebuffi Sylvestre-Alvise","unstructured":"Sylvestre-Alvise Rebuffi , Alexander Kolesnikov , Georg Sperl , and Christoph H. Lampert . 2017. iCaRL: Incremental classifier and representation learning . In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. 5533\u20135542 . Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. 2017. iCaRL: Incremental classifier and representation learning. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. 5533\u20135542."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1162\/089976601750264965"},{"key":"e_1_2_1_32_1","volume-title":"Proceedings of the 33st International Conference on Neural Information Processing Systems. 6\u201312","author":"Sohn Kihyuk","year":"2020","unstructured":"Kihyuk Sohn , David Berthelot , Chun-Liang Li , Zizhao Zhang , Nicholas Carlini , Ekin D. Cubuk , Alex Kurakin , Han Zhang , and Colin Raffel . 2020 . FixMatch: Simplifying semi-supervised learning with consistency and confidence . In Proceedings of the 33st International Conference on Neural Information Processing Systems. 6\u201312 . Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Han Zhang, and Colin Raffel. 2020. FixMatch: Simplifying semi-supervised learning with consistency and confidence. In Proceedings of the 33st International Conference on Neural Information Processing Systems. 6\u201312."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2019.00554"},{"key":"e_1_2_1_34_1","volume-title":"Proceedings of the 30th International Conference on Machine Learning, PMLR. 1139\u20131147","author":"Sutskever Ilya","unstructured":"Ilya Sutskever , James Martens , George E. Dahl , and Geoffrey E. Hinton . 2013. On the importance of initialization and momentum in deep learning . In Proceedings of the 30th International Conference on Machine Learning, PMLR. 1139\u20131147 . Ilya Sutskever, James Martens, George E. Dahl, and Geoffrey E. Hinton. 2013. On the importance of initialization and momentum in deep learning. In Proceedings of the 30th International Conference on Machine Learning, PMLR. 1139\u20131147."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/asaa027"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2019.00128"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.177"}],"container-title":["ACM Transactions on Knowledge Discovery from Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3468675","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3468675","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:17:18Z","timestamp":1750191438000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3468675"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,3]]},"references-count":36,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,4,30]]}},"alternative-id":["10.1145\/3468675"],"URL":"https:\/\/doi.org\/10.1145\/3468675","relation":{},"ISSN":["1556-4681","1556-472X"],"issn-type":[{"value":"1556-4681","type":"print"},{"value":"1556-472X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,3]]},"assertion":[{"value":"2020-10-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-05-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-09-03","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}