{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T16:07:10Z","timestamp":1772122030151,"version":"3.50.1"},"reference-count":29,"publisher":"Association for Computing Machinery (ACM)","issue":"5","license":[{"start":{"date-parts":[[2024,10,26]],"date-time":"2024-10-26T00:00:00Z","timestamp":1729900800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Higher Education Sprout Project by the Ministry of Education","award":["113L900901\/113L900902\/113L900903"],"award-info":[{"award-number":["113L900901\/113L900902\/113L900903"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2024,10,31]]},"abstract":"<jats:p>\n            In natural language processing (NLP), zero-shot topic classification requires machines to understand the contextual meanings of texts in a downstream task without using the corresponding labeled texts for training, which is highly desirable for various applications. In this article, we propose a novel approach to construct a zero-shot task-specific model called WC-SBERT with satisfactory performance. The proposed approach is highly efficient since it uses light self-training requiring target labels (target class names of downstream tasks) only, which is distinct from other research that uses both the target labels and the unlabeled texts for training. In particular, during the pre-training stage, WC-SBERT uses contrastive learning with multiple negative ranking losses to construct the pre-trained model based on the similarity between Wiki categories. For the self-training stage, online contrastive loss is utilized to reduce the distance between a target label and Wiki categories of similar Wiki pages to the label. Experimental results indicate that compared to existing self-training models, WC-SBERT achieves rapid inference on approximately 6.45 million Wiki text entries by utilizing pre-stored Wikipedia text embeddings, significantly reducing inference time per sample by a factor of 2,746 to 16,746. During the fine-tuning step, the time required for each sample is reduced by a factor of 23\u201367. Overall, the total training time shows a maximum reduction of 27.5 times across different datasets. Most importantly, our model has achieved state-of-the-art (SOTA) accuracy on two of the three commonly used datasets for evaluating zero-shot classification, namely the AG News (0.84) and Yahoo! Answers (0.64) datasets. The code for WC-SBERT is publicly available on GitHub,\n            <jats:xref ref-type=\"fn\">\n              <jats:sup>1<\/jats:sup>\n            <\/jats:xref>\n            and the dataset can also be accessed on Hugging Face.\n            <jats:xref ref-type=\"fn\">\n              <jats:sup>2<\/jats:sup>\n            <\/jats:xref>\n          <\/jats:p>","DOI":"10.1145\/3678183","type":"journal-article","created":{"date-parts":[[2024,7,18]],"date-time":"2024-07-18T13:11:55Z","timestamp":1721308315000},"page":"1-18","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["WC-SBERT: Zero-Shot Topic Classification Using SBERT and Light Self-Training on Wikipedia Categories"],"prefix":"10.1145","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-7438-2782","authenticated-orcid":false,"given":"Te-Yu","family":"Chi","sequence":"first","affiliation":[{"name":"Department of CSIE, National Taiwan University, Taipei, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7319-9095","authenticated-orcid":false,"given":"Jyh-Shing Roger","family":"Jang","sequence":"additional","affiliation":[{"name":"Department of CSIE, National Taiwan University, Taipei, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,10,26]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-76298-0_52"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/3555776.3577653"},{"key":"e_1_3_3_4_2","first-page":"830","volume-title":"Proceedings of the 23rd National Conference on Artificial Intelligence (AAAI \u201908)","volume":"2","author":"Chang Ming-Wei","year":"2008","unstructured":"Ming-Wei Chang, Lev Ratinov, Dan Roth, and Vivek Srikumar. 2008. Importance of Semantic Representation: Dataless Classification. In Proceedings of the 23rd National Conference on Artificial Intelligence (AAAI \u201908), Vol. 2, AAAI Press, 830\u2013835."},{"key":"e_1_3_3_5_2","unstructured":"Qianben Chen Richong Zhang Yaowei Zheng and Yongyi Mao. 2022. Dual contrastive learning: Text classification via label-aware data augmentation. arXiv:2201.08702. Retrieved from https:\/\/arxiv.org\/abs\/2201.08702"},{"key":"e_1_3_3_6_2","unstructured":"Zewei Chu Karl Stratos and Kevin Gimpel. 2020. NatCat: Weakly supervised text classification with naturally annotated datasets. arXiv:2009.14335. Retrieved from https:\/\/arxiv.org\/abs\/2009.14335"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1423"},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.naacl-demo.10"},{"key":"e_1_3_3_9_2","doi-asserted-by":"crossref","first-page":"1107","DOI":"10.18653\/v1\/2022.emnlp-main.73","volume-title":"Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing","author":"Gera Ariel","year":"2022","unstructured":"Ariel Gera, Alon Halfon, Eyal Shnarch, Yotam Perlitz, Liat Ein-Dor, and Noam Slonim. 2022. Zero-Shot Text Classification with Self-Training. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. ACL, 1107\u20131119. Retrieved from https:\/\/aclanthology.org\/2022.emnlp-main.73"},{"key":"e_1_3_3_10_2","unstructured":"Matthew Henderson Rami Al-Rfou Brian Strope Yun-Hsuan Sung L\u00e1szl\u00f3 Luk\u00e1cs Ruiqi Guo Sanjiv Kumar Balint Miklos and Ray Kurzweil. 2017. Efficient natural language response suggestion for smart reply. arXiv:1705.00652. Retrieved from https:\/\/arxiv.org\/abs\/1705.00652"},{"key":"e_1_3_3_11_2","unstructured":"Chaoqun Liu Wenxuan Zhang Guizhen Chen Xiaobao Wu Anh Tuan Luu Chip Hong Chang and Lidong Bing. 2023. Zero-shot text classification via self-supervised tuning. arXiv:2305.11442. Retrieved from https:\/\/arxiv.org\/abs\/2305.11442"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-short.8"},{"key":"e_1_3_3_13_2","unstructured":"Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy Mike Lewis Luke Zettlemoyer and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv:1907.11692. Retrieved from http:\/\/arxiv.org\/abs\/1907.11692"},{"key":"e_1_3_3_14_2","doi-asserted-by":"crossref","unstructured":"Amil Merchant Elahe Rahimtoroghi Ellie Pavlick and Ian Tenney. 2020. What happens to BERT embeddings during fine-tuning? arXiv:2004.14448. Retrieved from https:\/\/arxiv.org\/abs\/2004.14448","DOI":"10.18653\/v1\/2020.blackboxnlp-1.4"},{"key":"e_1_3_3_15_2","volume-title":"Proceedings of the Workshop at ICLR 2013","author":"Mikolov Tomas","year":"2013","unstructured":"Tomas Mikolov, Kai Chen, G. S. Corrado, and Jeffrey Dean. 2013. Efficient Estimation of Word Representations in Vector Space. In Proceedings of the Workshop at ICLR 2013. Retrieved from https:\/\/www.researchgate.net\/publication\/234131319_Efficient_Estimation_of_Word_Representations_in_Vector_Space"},{"key":"e_1_3_3_16_2","first-page":"20","volume-title":"Proceedings of the International Conference on Document Analysis and Recognition","author":"Pe\u00f1a Alejandro","year":"2023","unstructured":"Alejandro Pe\u00f1a, Aythami Morales, Julian Fierrez, Ignacio Serna, Javier Ortega-Garcia, I\u00f1igo Puente, Jorge Cordova, and Gonzalo Cordova. 2023. Leveraging large language models for topic classification in the domain of public affairs. In Proceedings of the International Conference on Document Analysis and Recognition. Springer, 20\u201333."},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162"},{"key":"e_1_3_3_18_2","unstructured":"Alec Radford Jeff Wu Rewon Child David Luan Dario Amodei and Ilya Sutskever. 2019. Language Models are Unsupervised Multitask Learners. Retrieved from https:\/\/d4mucfpksywv.cloudfront.net\/better-language-models\/language-models.pdf"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1410"},{"key":"e_1_3_3_20_2","doi-asserted-by":"crossref","unstructured":"Timo Schick and Hinrich Sch\u00fctze. 2020. Exploiting cloze questions for few shot text classification and natural language inference. arXiv:2001.07676. Retrieved from https:\/\/arxiv.org\/abs\/2001.07676","DOI":"10.18653\/v1\/2021.eacl-main.20"},{"key":"e_1_3_3_21_2","unstructured":"Sonish Sivarajkumar and Yanshan Wang. 2022. HealthPrompt: A zero-shot learning paradigm for clinical natural language processing. arXiv:2203.05061. Retrieved from https:\/\/arxiv.org\/abs\/2203.05061"},{"key":"e_1_3_3_22_2","doi-asserted-by":"crossref","first-page":"7508","DOI":"10.18653\/v1\/2022.emnlp-main.509","volume-title":"Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing","author":"Kar Mozes van de","year":"2022","unstructured":"Mozes van de Kar, Mengzhou Xia, Danqi Chen, and Mikel Artetxe. 2022. Don\u2019t Prompt, Search! Mining-based Zero-Shot Learning with Language Models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. ACL, 7508\u20137520. DOI: https:\/\/aclanthology.org\/2022.emnlp-main.509"},{"key":"e_1_3_3_23_2","doi-asserted-by":"crossref","first-page":"30","DOI":"10.18653\/v1\/2023.acl-srw.4","volume-title":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop)","volume":"4","author":"Wang Yuki","year":"2023","unstructured":"Yuki Wang, Wei Wang, Qi Chen, Kaizhu Huang, Anh Nguyen, and Suparna De. 2023. Prompt-based Zero-shot Text Classification with Conceptual Knowledge. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop), Vol. 4, ACL, 30\u201338."},{"key":"e_1_3_3_24_2","doi-asserted-by":"crossref","first-page":"7042","DOI":"10.18653\/v1\/2022.emnlp-main.474","volume-title":"Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing","author":"Yang Ping","year":"2022","unstructured":"Ping Yang, Junjie Wang, Ruyi Gan, Xinyu Zhu, Lin Zhang, Ziwei Wu, Xinyu Gao, Jiaxing Zhang, and Tetsuya Sakai. 2022. Zero-Shot Learners for Natural Language Understanding via a Unified Multiple Choice Perspective. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. ACL, 7042\u20137055. https:\/\/aclanthology.org\/2022.emnlp-main.474"},{"key":"e_1_3_3_25_2","unstructured":"Xiangli Yang Zixing Song Irwin King and Zenglin Xu. 2021. A survey on deep semi-supervised learning. arXiv:2103.00550. Retrieved from https:\/\/arxiv.org\/abs\/2103.00550"},{"key":"e_1_3_3_26_2","volume-title":"Advances in Neural Information Processing Systems","author":"Yang Zhilin","year":"2019","unstructured":"Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R. Salakhutdinov, and Quoc V. Le. 2019. XLNet: Generalized Autoregressive Pretraining for Language Understanding. In Advances in Neural Information Processing Systems. H. Wallach, H. Larochelle, A. Beygelzimer, F. d\u2019Alch\u00e9-Buc, E. Fox, and R. Garnett (Eds.), Vol. 32, Curran Associates, Inc. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2019\/file\/dc6a7e655d7e5840e66733e9ee67cc69-Paper.pdf"},{"key":"e_1_3_3_27_2","first-page":"17283","volume-title":"Advances in Neural Information Processing Systems","author":"Zaheer Manzil","year":"2020","unstructured":"Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, and Amr Ahmed. 2020. Big Bird: Transformers for Longer Sequences. In Advances in Neural Information Processing Systems. H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (Eds.), Vol. 33, Curran Associates, Inc., 17283\u201317297. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2020\/file\/c8512d142a2d849725f31a9a7a361ab9-Paper.pdf"},{"key":"e_1_3_3_28_2","unstructured":"Ruohong Zhang Yau-Shian Wang and Yiming Yang. 2023. Generation-driven contrastive self-training for zero-shot text classification with instruction-tuned GPT. arXiv:2304.11872. Retrieved from https:\/\/arxiv.org\/abs\/2304.11872"},{"key":"e_1_3_3_29_2","volume-title":"Advances in Neural Information Processing Systems","author":"Zhang Xiang","year":"2015","unstructured":"Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-Level Convolutional Networks for Text Classification. In Advances in Neural Information Processing Systems. C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett (Eds.), Vol. 28, Curran Associates, Inc. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2015\/file\/250cf8b51c773f3f8dc8b4be867a9a02-Paper.pdf"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01219-9_18"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3678183","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3678183","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:54:08Z","timestamp":1750287248000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3678183"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,26]]},"references-count":29,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2024,10,31]]}},"alternative-id":["10.1145\/3678183"],"URL":"https:\/\/doi.org\/10.1145\/3678183","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"value":"2157-6904","type":"print"},{"value":"2157-6912","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,26]]},"assertion":[{"value":"2024-01-04","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-07-06","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-10-26","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}