{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:19:48Z","timestamp":1750220388050,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":30,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,8,14]],"date-time":"2021-08-14T00:00:00Z","timestamp":1628899200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["IIS-19-56151,IIS-17-41317,IIS-17-04532,2019897"],"award-info":[{"award-number":["IIS-19-56151,IIS-17-41317,IIS-17-04532,2019897"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000185","name":"Defense Advanced Research Projects Agency","doi-asserted-by":"publisher","award":["FA8750-19-2-1004,W911NF-17-C-0099"],"award-info":[{"award-number":["FA8750-19-2-1004,W911NF-17-C-0099"]}],"id":[{"id":"10.13039\/100000185","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,8,14]]},"DOI":"10.1145\/3447548.3470810","type":"proceedings-article","created":{"date-parts":[[2021,8,12]],"date-time":"2021-08-12T06:12:03Z","timestamp":1628748723000},"page":"4052-4053","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["On the Power of Pre-Trained Text Representations"],"prefix":"10.1145","author":[{"given":"Yu","family":"Meng","sequence":"first","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiaxin","family":"Huang","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yu","family":"Zhang","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiawei","family":"Han","sequence":"additional","affiliation":[{"name":"University of Illinois at Urbana-Champaign, Urbana, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,8,14]]},"reference":[{"key":"e_1_3_2_1_1_1","unstructured":"David M. Blei Thomas L. Griffiths Michael I. Jordan and Joshua B. Tenenbaum. 2003 a. Hierarchical Topic Models and the Nested Chinese Restaurant Process. In NIPS . David M. Blei Thomas L. Griffiths Michael I. Jordan and Joshua B. Tenenbaum. 2003 a. Hierarchical Topic Models and the Nested Chinese Restaurant Process. In NIPS ."},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"crossref","unstructured":"David M. Blei Andrew Y. Ng and Michael I. Jordan. 2003 b. Latent Dirichlet Allocation. In J. Mach. Learn. Res. David M. Blei Andrew Y. Ng and Michael I. Jordan. 2003 b. Latent Dirichlet Allocation. In J. Mach. Learn. Res.","DOI":"10.7551\/mitpress\/1120.003.0082"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00051"},{"key":"e_1_3_2_1_4_1","unstructured":"Ming-Wei Chang Lev-Arie Ratinov Dan Roth and Vivek Srikumar. 2008. Importance of Semantic Representation: Dataless Classification.. In AAAI . Ming-Wei Chang Lev-Arie Ratinov Dan Roth and Vivek Srikumar. 2008. Importance of Semantic Representation: Dataless Classification.. In AAAI ."},{"key":"e_1_3_2_1_5_1","volume-title":"Manning","author":"Clark Kevin","year":"2020","unstructured":"Kevin Clark , Minh-Thang Luong , Quoc V. Le , and Christopher D . Manning . 2020 . ELECTRA : Pre-training Text Encoders as Discriminators Rather Than Generators. In ICLR . Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020. ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators. In ICLR ."},{"key":"e_1_3_2_1_6_1","volume-title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT .","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . 2019 . BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT . Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL-HLT ."},{"key":"e_1_3_2_1_7_1","unstructured":"Xiaotao Gu Zihan Wang Zhenyu Bi Yu Meng Liyuan Liu Jiawei Han and Jingbo Shang. 2021. UCPhrase: Unsupervised Context-aware Quality Phrase Tagging. In KDD . Xiaotao Gu Zihan Wang Zhenyu Bi Yu Meng Liyuan Liu Jiawei Han and Jingbo Shang. 2021. UCPhrase: Unsupervised Context-aware Quality Phrase Tagging. In KDD ."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"crossref","unstructured":"Jiaxin Huang Yu Meng Fang Guo Heng Ji and Jiawei Han. 2020 a. Weakly-Supervised Aspect-Based Sentiment Analysis via Joint Aspect-Sentiment Topic Embedding. In EMNLP . Jiaxin Huang Yu Meng Fang Guo Heng Ji and Jiawei Han. 2020 a. Weakly-Supervised Aspect-Based Sentiment Analysis via Joint Aspect-Sentiment Topic Embedding. In EMNLP .","DOI":"10.18653\/v1\/2020.emnlp-main.568"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"crossref","unstructured":"Jiaxin Huang Yiqing Xie Yu Meng Yunyi Zhang and Jiawei Han. 2020 b. CoRel: Seed-Guided Topical Taxonomy Construction by Concept Learning and Relation Transferring. In KDD . Jiaxin Huang Yiqing Xie Yu Meng Yunyi Zhang and Jiawei Han. 2020 b. CoRel: Seed-Guided Topical Taxonomy Construction by Concept Learning and Relation Transferring. In KDD .","DOI":"10.1145\/3394486.3403244"},{"key":"e_1_3_2_1_10_1","volume-title":"RoBERTa: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692","author":"Liu Yinhan","year":"2019","unstructured":"Yinhan Liu , Myle Ott , Naman Goyal , Jingfei Du , Mandar Joshi , Danqi Chen , Omer Levy , Mike Lewis , Luke Zettlemoyer , and Veselin Stoyanov . 2019. RoBERTa: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 ( 2019 ). Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"crossref","unstructured":"Yu Meng Jiaxin Huang Guangyuan Wang Zihan Wang Chao Zhang Yu Zhang and Jiawei Han. 2020 a. Discriminative Topic Mining via Category-Name Guided Text Embedding. In WWW . Yu Meng Jiaxin Huang Guangyuan Wang Zihan Wang Chao Zhang Yu Zhang and Jiawei Han. 2020 a. Discriminative Topic Mining via Category-Name Guided Text Embedding. In WWW .","DOI":"10.1145\/3366423.3380278"},{"key":"e_1_3_2_1_12_1","unstructured":"Yu Meng Jiaxin Huang Guangyuan Wang Chao Zhang Honglei Zhuang Lance M. Kaplan and Jiawei Han. 2019 a. Spherical Text Embedding. In NeurIPS . Yu Meng Jiaxin Huang Guangyuan Wang Chao Zhang Honglei Zhuang Lance M. Kaplan and Jiawei Han. 2019 a. Spherical Text Embedding. In NeurIPS ."},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"crossref","unstructured":"Yu Meng Jiaming Shen Chao Zhang and Jiawei Han. 2018. Weakly-Supervised Neural Text Classification. In CIKM . Yu Meng Jiaming Shen Chao Zhang and Jiawei Han. 2018. Weakly-Supervised Neural Text Classification. In CIKM .","DOI":"10.1145\/3269206.3271737"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"crossref","unstructured":"Yu Meng Jiaming Shen Chao Zhang and Jiawei Han. 2019 b. Weakly-Supervised Hierarchical Text Classification. In AAAI . Yu Meng Jiaming Shen Chao Zhang and Jiawei Han. 2019 b. Weakly-Supervised Hierarchical Text Classification. In AAAI .","DOI":"10.1145\/3269206.3271737"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"crossref","unstructured":"Yu Meng Yunyi Zhang Jiaxin Huang Chenyan Xiong Heng Ji Chao Zhang and Jiawei Han. 2020 b. Text Classification Using Label Names Only: A Language Model Self-Training Approach. In EMNLP . Yu Meng Yunyi Zhang Jiaxin Huang Chenyan Xiong Heng Ji Chao Zhang and Jiawei Han. 2020 b. Text Classification Using Label Names Only: A Language Model Self-Training Approach. In EMNLP .","DOI":"10.18653\/v1\/2020.emnlp-main.724"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"crossref","unstructured":"Yu Meng Yunyi Zhang Jiaxin Huang Yu Zhang Chao Zhang and Jiawei Han. 2020 c. Hierarchical Topic Mining via Joint Spherical Tree and Text Embedding. In KDD . Yu Meng Yunyi Zhang Jiaxin Huang Yu Zhang Chao Zhang and Jiawei Han. 2020 c. Hierarchical Topic Mining via Joint Spherical Tree and Text Embedding. In KDD .","DOI":"10.1145\/3394486.3403242"},{"key":"e_1_3_2_1_17_1","unstructured":"Tomas Mikolov Ilya Sutskever Kai Chen Gregory S. Corrado and Jeffrey Dean. 2013. Distributed Representations of Words and Phrases and their Compositionality. In NIPS . Tomas Mikolov Ilya Sutskever Kai Chen Gregory S. Corrado and Jeffrey Dean. 2013. Distributed Representations of Words and Phrases and their Compositionality. In NIPS ."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"crossref","unstructured":"David M. Mimno Wei Li and Andrew McCallum. 2007. Mixtures of hierarchical topics with Pachinko allocation. In ICML . David M. Mimno Wei Li and Andrew McCallum. 2007. Mixtures of hierarchical topics with Pachinko allocation. In ICML .","DOI":"10.1145\/1273496.1273576"},{"key":"e_1_3_2_1_19_1","unstructured":"Maximilian Nickel and Douwe Kiela. 2017. Poincar\u00e9 Embeddings for Learning Hierarchical Representations. In NIPS . Maximilian Nickel and Douwe Kiela. 2017. Poincar\u00e9 Embeddings for Learning Hierarchical Representations. In NIPS ."},{"key":"e_1_3_2_1_20_1","volume-title":"Manning","author":"Pennington Jeffrey","year":"2014","unstructured":"Jeffrey Pennington , Richard Socher , and Christopher D . Manning . 2014 . GloVe: Global Vectors for Word Representation. In EMNLP . Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global Vectors for Word Representation. In EMNLP ."},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"crossref","unstructured":"Matthew E. Peters Mark Neumann Mohit Iyyer Matt Gardner Christopher Clark Kenton Lee and Luke Zettlemoyer. 2018. Deep contextualized word representations. In NAACL-HLT . Matthew E. Peters Mark Neumann Mohit Iyyer Matt Gardner Christopher Clark Kenton Lee and Luke Zettlemoyer. 2018. Deep contextualized word representations. In NAACL-HLT .","DOI":"10.18653\/v1\/N18-1202"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"crossref","unstructured":"Jiaming Shen Wenda Qiu Yu Meng Jingbo Shang Xiang Ren and Jiawei Han. 2021. TaxoClass: Hierarchical Multi-Label Text Classification Using Only Class Names. In NAACL-HLT . Jiaming Shen Wenda Qiu Yu Meng Jingbo Shang Xiang Ren and Jiawei Han. 2021. TaxoClass: Hierarchical Multi-Label Text Classification Using Only Class Names. In NAACL-HLT .","DOI":"10.18653\/v1\/2021.naacl-main.335"},{"key":"e_1_3_2_1_23_1","unstructured":"Suzanna Sia Ayush Dalmia and Sabrina J Mielke. 2020. Tired of Topic Models? Clusters of Pretrained Word Embeddings Make for Fast and Good Topics too!. In EMNLP . Suzanna Sia Ayush Dalmia and Sabrina J Mielke. 2020. Tired of Topic Models? Clusters of Pretrained Word Embeddings Make for Fast and Good Topics too!. In EMNLP ."},{"key":"e_1_3_2_1_24_1","unstructured":"Alexandru Tifrea Gary B\u00e9cigneul and Octavian-Eugen Ganea. 2019. Poincar\u00e9 GloVe: Hyperbolic Word Embeddings. In ICLR . Alexandru Tifrea Gary B\u00e9cigneul and Octavian-Eugen Ganea. 2019. Poincar\u00e9 GloVe: Hyperbolic Word Embeddings. In ICLR ."},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"crossref","unstructured":"Zihan Wang Dheeraj Mekala and Jingbo Shang. 2021. X-Class: Text Classification with Extremely Weak Supervision. In NAACL-HLT . Zihan Wang Dheeraj Mekala and Jingbo Shang. 2021. X-Class: Text Classification with Extremely Weak Supervision. In NAACL-HLT .","DOI":"10.18653\/v1\/2021.naacl-main.242"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"crossref","unstructured":"Sam Wiseman and Karl Stratos. 2019. Label-Agnostic Sequence Labeling by Copying Nearest Neighbors. In ACL . Sam Wiseman and Karl Stratos. 2019. Label-Agnostic Sequence Labeling by Copying Nearest Neighbors. In ACL .","DOI":"10.18653\/v1\/P19-1533"},{"key":"e_1_3_2_1_27_1","volume-title":"Le","author":"Yang Zhilin","year":"2019","unstructured":"Zhilin Yang , Zihang Dai , Yiming Yang , Jaime G. Carbonell , Ruslan Salakhutdinov , and Quoc V . Le . 2019 . XLNet: Generalized Autoregressive Pretraining for Language Understanding. In NeurIPS . Zhilin Yang, Zihang Dai, Yiming Yang, Jaime G. Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2019. XLNet: Generalized Autoregressive Pretraining for Language Understanding. In NeurIPS ."},{"key":"e_1_3_2_1_28_1","unstructured":"Jingqing Zhang Yao Zhao Mohammad Saleh and Peter J. Liu. 2020 b. PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization. In ICML . Jingqing Zhang Yao Zhao Mohammad Saleh and Peter J. Liu. 2020 b. PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization. In ICML ."},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"crossref","unstructured":"Yu Zhang Yu Meng Jiaxin Huang Frank F Xu Xuan Wang and Jiawei Han. 2020 a. Minimally supervised categorization of text with metadata. In SIGIR . Yu Zhang Yu Meng Jiaxin Huang Frank F Xu Xuan Wang and Jiawei Han. 2020 a. Minimally supervised categorization of text with metadata. In SIGIR .","DOI":"10.1145\/3397271.3401168"},{"key":"e_1_3_2_1_30_1","volume-title":"MATCH: Metadata-Aware Text Classification in A Large Hierarchy. In WWW .","author":"Zhang Yu","year":"2021","unstructured":"Yu Zhang , Zhihong Shen , Yuxiao Dong , Kuansan Wang , and Jiawei Han . 2021 . MATCH: Metadata-Aware Text Classification in A Large Hierarchy. In WWW . Yu Zhang, Zhihong Shen, Yuxiao Dong, Kuansan Wang, and Jiawei Han. 2021. MATCH: Metadata-Aware Text Classification in A Large Hierarchy. In WWW ."}],"event":{"name":"KDD '21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","sponsor":["SIGMOD ACM Special Interest Group on Management of Data","SIGKDD ACM Special Interest Group on Knowledge Discovery in Data"],"location":"Virtual Event Singapore","acronym":"KDD '21"},"container-title":["Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3447548.3470810","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3447548.3470810","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3447548.3470810","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:18:32Z","timestamp":1750191512000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3447548.3470810"}},"subtitle":["Models and Applications in Text Mining"],"short-title":[],"issued":{"date-parts":[[2021,8,14]]},"references-count":30,"alternative-id":["10.1145\/3447548.3470810","10.1145\/3447548"],"URL":"https:\/\/doi.org\/10.1145\/3447548.3470810","relation":{},"subject":[],"published":{"date-parts":[[2021,8,14]]},"assertion":[{"value":"2021-08-14","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}